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(57) Abstract 



A novel method and apparatus for protection of streamed media content is disclosed. The apparatus includes control means for 
governance of content streams or objects, decryption means for decrypting content streams or objects under control of the control means^ 
and feedback means for tracking actual use of content streams or objects. The control means may operate in accordance with rules received 
as part of the streamed content, or through a side-band channel. 'Ilie rules may specify allowed uses of the content, including whether or 
not the content can be copied or transferred, and whether and under what circumstances received content may t>e "checked out" of one 
device and used in a second device, fhe rules may also include or specify budgets, and a requirement that audit information be collected 
and/or transmitted to an external server. The apparatus may include a media player designed to call plugins to assist in rendering content. 
A "trust piugin" and its use are disclosed so that a media player designed for use with unprotected content may render protected content 
without the necessity of requiring any changes to the media player. The streamed content may be in a number of different formats, including 
MPEG-4, MP3, and the RMFF format. 
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HELD OF THF I^f\^^^JTTf^p^ 

"-=l-esge„e.„y,oco™p„,e,„^„,e>ec.o„csecrt,y. More 
panicularly, aiisinvennonrda.es to systems a„d,„„h~. r ^ "ore 

s-reanted fomta,. '""^ '"'"^'ion tn 

BACKGRflimn 

.-=s:e~re::ir^^^^^ 

lor several adopted, proposed or de facto standards Th« 
copied and improperly disseminated, and the consequent reluctance of 

can be protected. " ' ^^^^ ^'g"^' strean^s 

SUMMARY OFTJjEJjWElNOT^ 

Consistent with the inventmr, rh;,. 

protection of infom... T ^f'^"^"^"^" '^--b- ^ new architecture for 

piuieciion ot information provided in stre;impH fn™n. -ru- 

context of a genenc system K ""™'=^ ^l"'' "chtteemre is described m the 

.0 the MPEG 4 sT f " ""^ -"""i P-"™ 

to me MFtG-4 specification (ISO/IEC 1 44Qfi i ^ th^. i, u h 

with the proviso that the descnbed ' 

respects. A vanety o d f^^t T "'^^ '""^ ^^^^^^^ 
.1 . of different embodiments ,s descnbed, including an MPEG-4 

.peets^rr:::::— 

content proteetton fitnettonalit^T ! , / """" "f 

«^ct,ona,i.yton,o!:;tr^;:::~ 

tncotporatton of rule/eonnv,, r "°" '™ "^""'^ '"^"«"= 

content tL^l" 7' T— — ■ P™ec.ion of 

ougn mechamsms such as encryption and watermarking. 
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Systems and methods consistent with the present invention perfonn content 
protecnon and digital nghts management. A streammg media player consistent with the 
present mvention includes a port designed to accept a digital bit stream. The digital bh 
stream includes content, wWch is enc^ted at least in part, and a secure container 
.ncludmg control mforma.on des.^ed to control use of the content, mc.udmg at least one 
key su,table for decryption of at least a portion of the content. The media player also 
mcludes a control arrangement mcludmg a means for opening secure containers and 
extractmg cryptographic keys, and .cans for decryptmg the encrypted portion of the 
content. 

BRIEF D ESCRIPTION OF THK PR AWiMr^c 

The accompanying drawings, which are incorporated in and constitute a part of this 
spec.ficat.on, illustrate an embodiment of the mvention and, together with the descnption 
serve to explain the advantages and pnnc.ples of the mvention. In the drawings 
FIG. 1 shows a genenc system consistent with the presem invention- 
FIG. 2 shows an exemplai^ Header 201 consistent w.th the present invention- 
FIG. 3 shows a general encoding fonnat consistent with the present invention- 
FIG. 4 Illustrates one manner for storing a representation of a work consistent with 
the present invention; 

FIG. 5 shows an example of a control message format; 

FIG. 6 is a flow diagram illustrating one embodiment of the steps which take place 
using the functional blocks of FIG. 1 ; 

FIG. 7 niustrates a form wherein the control messages may be stored in Control 
olocK 13; 

FIG. 8 shows MPEG-4 System 801 consistent w.th the present invention; 
FIG. 9 shows an example of a message format; 

FIG. 10 Illustrates an IPMP table consistent with the present invention; 
FIG. 1 1 Illustrates a system consistent with the present invention; 
FIG. 12 illustrates one embodiment of the DigiBox format; 
FIG. 13 shows an example of a Real Networks file format (RMFF)- 
FIG. 14 shows an RNPFF format consistent with the present inventlon- 
FIG. 15 illustrates the flow of changes to data m the Real Networks file fomiat , 
an architecture consistent with the present invention; 

FIG. 1 6 illustrates a standard Real Networks architecture; 

FIG 17 shows an exemplary architecture in which a trxist plugin operates within the 
overall Real Networks architecture; 



tm 
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FIG. 1 9 shows „„= embodtaem of protecion applied ,o ,h= MPa fo™„; 
FIG. 20 ,ilus,r„es one cmbodimen, of a„ MP3 player designed » process a„d 
render proiccledcontenl; "process and 

FIG. 2 1 illns,ra,es ,he How of da,a ,„ one , 
4 file may be crea.edcons,s.en. With .he present invne.,o„- 

inco„ ""."'"""""""""""f^"'""— ^™™.nwhichcon.ol™aybe 
.ncon>o.,ed ,„.o an e.,s,ing MPBG-4 strean, co„s,s.en, wi.h „,e presen, ,„ve„, J 

na s o»s a system co„s,s,e„. „„h ,he principles of ,he presen. ,nven.ion; 
FIG. 2 s „„s a sys.em co„sis.e„, „i,|, .he p,i„e,ples of d,e p.se„. inveni.on 

FIG. 26 iMus.ra,es a Header CMPO 2601 co„s,s.en. wuh .he presen. .„v.„„„n; 

.he pnnc't; fir ^"^"^ '~ °'>^^ — 

me pnncipies of the present invention; and 

DETAII rn PESrHH Tifym 

Reference w,ll now be made in deiail ,o impleme„.a„ons cons.slen. wi.h ,he 
pnncples „ ,he presen, ,nven.,on . Ill^s^ed m ,he aecon,pa„y,„g drawngs. 

ass, J '"'"'^ °f -i^ed .0 .he 

^s gnee of he curren, app„ca..on, are hereby inco^ra,ed ,„ .he.r emireiy by refe^nce- 
G.n er, e. al.. "Sys-ems and Me.hods for Secure Transacnon Managenren. and E ,c 

r " ^'^^-^-'^ • Process 

Co„„l A„.oma.,on, Dis.r.ba.ed Comp„.,„g. and ,^gh.s Managemen,. " US Pa.en. 

Apphca.,o„ Senal No. Om^,U. f.led on A„,« 12, l« rGln.er .7,2") Van W,e e. 

^, Sicganograp^c Techn.,„es for Securely Del.venng Elecronic Digi«. R.gh.s ' 

Managemen. Infon.a.,on Over W„e Comm^.catas Chamiels, U S Pa,em 

^ptaa.,o„ Senal No. 08/689.606. flied on Aug« n. ,996 ("V^ Wle^ ; Ginier e, al 

No. 08/706 206. filed on A„g« 30, 1996 (-G.n.er, •206",; Shear, e. al, "C.yp.o^aphic 
MeU,ods, Appara-ns S. Sys.ems for S.orage Med.a Elec»nic Righ,s Managlem in 
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Closed & Connected Appliances, " U.S. Patent Application Senal No. 08/848,077 filed on 
May 1 5, 1997 ("Shear"); Collberg et al, "Obfuscatton Techniques for Enhancing Software 
Secunty, " U.S. Patent Application Serial No. 09/095,346, filed on June 9 1 998 
("Collberg"); Shear, "Database Usage Metering and Protection System and Method " U S 
Patent No. 4,827,508, issued on May 2, 1989 ("Shear Patent"). 

FIG. I illustrates Media System 1, which is capable of accepting, decoding and 
rendenng streamed multimedia content. This is a generic system, though it includes 
elements based on the MPEG-4 specification. Media System 1 may include software 
modules, hardware (including integrated circuits) or a combination. In one embodiment 
Media System 1 may include a Protected Processing Environment (PPE) as described in' 
the Ginter '333 application. 

In FIG. 1, Bit Stream 2 represents input information received by System 1 Bit 
Stream 2 may be received through a connection to an external network (e.g., an Internet 
connection, a cable hookup, radio transmission from a satellite broadcaster, etc.), or may be 
received from a portable memory device, such as a DVD player. 

Bit Stream 2 is made up of a group of related streams of information, mcluding 
Organization Stream 3, Audio Stream 4, Video Stream 5. Control Stream 6, and Info 
Sfream 31. Each of these streams is encoded into the overall Bit Stream 2. Each of these 
represents a category of streams, so that, for example, Video Stream 5 may be made up of a 
number of separate Video Streams. 

These streams correspond generally to streams descnbed in the MPEG-4 format as 

follows: 

Orgamzation Stream 3 corresponds generally to the BIFS stream and the OD 
("Object Descriptor") stream. 

Audio Stream 4 and Video Stream 5 correspond generally to the Audio and Video 
streams. 

Control Stream 6 corresponds generally to the IPMP stream. 

Audio Stream 4 includes compressed (and possibly encrypted) digital audio 
information. This information ,s used to create the sound rendered and output by Media 
System 1. Audio Sfream 1 may represent multiple audio sfreams. These multiple streams 
may ac, together to make up the audio output, or may represent alternative audio outputs. 

Video Sfream 5 includes compressed (and possibly encrypted) digital video 
mformation. This information is used to create the images and video rendered and output 
by Media System 1. Video Stream 5 may represem multiple video streams. These 
multiple streams may act together to make up the video output, or may represent alternative 
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video outputs. 

con,., s.e^ , c„r 

Info Strean, 3 1 carries additional mfonnafon associated with the content in oth 

(e g., a cable, a bus an .nfrared .Tj ' '=''ann.l 

^.^a.. paces need , t In^^ ^^^^^ "^''^ 

en-bod,.en., eac. pac.e. ..y ,„eMe ,nd,v,dna, s,l .r::: " 
^Cow. Heade, 20, ,nc,Je. ^^2^ l^', 

- a header. F.e,d 203 .denM ,1 ^ ' '"""'^'"^ 

™'"'"P'"""'".ypc<>fs«ani (e.g.. Audio Slrram 

-a^s. ,nc,„d,„, .endenn, „L .,::zz:z:>7T:r " 

specity an elapsed time from commencement of rendenng, 
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and Composite Block 1 1 may use that elapsed time to determine when to render the 
associated object 

Field 205 contains a Governance Indicator. Field 206 identifies the number of 
following packets which are part of the identified stream. In each case, the relevant 
mformation is encodedm a bmary format. For example, Field 202 might include an 
arbitrary sequence of bits which is recognized as indicating a header, and Field 203 might 
include two bits, thereby allowing encoding of four different stream types. 

Returning to FIG. 1 , System 1 includes Demux 7, which accepts as input Bit Stream 
2 and routes individual streams (sometimes referred to as Elementary Streams or "ESs") to 
appropriate functional blocks of the system. 

Bit Stream 2 may be encoded in the format illustrated in FIG. 3. In this figure, 
Header 301 is encountered in the bit stream, with Packet 302 following, and so on through 
Packet 308. 

When Demux 7 encounters Header 301, Demux 7 identifies Header 301 as a header 
and uses the header information to identify Packets 302-305 as organization stream 
packets. Demux 7 uses this information to route these packets to Organization Block 8. 
Demux 7 handles Header 306 m a similar mamier, using the contained information to route 
Packets 307 and 308 to AV ("Audio Video") Block 9. 

AV Block 9 includes Decompressor 10, which accepts Elementary Streams from 
Audio Stream 4 and Video Stream 5 and decompresses those streams. As decompressed, 
the stream infomiat.on is placed in a format which allows it to be manipulated and output 
(through a video display, speakers, etc.). If multiple streams exist (e.g:, two video streams 
each describmg an aspect of a video sequence), AV Block 9 uses the ES_ID to assign each 
packet to the appropriate stream. 

Organization Block 8 stores pointer information identifying particular audio 
streams and video streams contamed in a particular object, as well as metadata mformation 
describing, for example, where the object is located, when it is to be displayed (e.g., the 
time stamp associated w.lh the object), and its relationship to other objects (e.g., is one 
video object in front of or behind another video object). This organization may be 
maintained hierarchically, with individual streams represemed at the lowest level, 
groupings of streams into objects at a higher level, complete scenes at a still highlr level, 
and the entire work at the highest level. 

FIG. 4 illusfrates one manner in which Organization Block 8 may store a 
representation of a work. In this Figure, Tree 401 represents an emire audiovisual work. 
Branch 402 represents a high-level organization of the work. This may include, for 
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Nodel '11 ■ range, wh„e 

* rep,«se„B a ttee ,mm=d,a.ely behind one of the characters 

^^^^ 

organ,zation and reIat.onsh.p of those elements C " 

H ui mose elements. Composite Block 1 1 accent.: 
decompressed audiovisual objects from AV Bloot o . 

specified hvinf ^ 9, and organizes those objects as 

specified by mformat.on from Orgamzation Block 8. Composite Block II th. u 

is Jo He;;, Z : r^'- ~ P-™- -^he strean, wh.ch 

y mis confroi message (this may indicate that the entiretv of th. ch- 

FIG. 3. tathe exa„,p,e .ho™. Control Message 50, catries the ™,„e ID ,, lOM, 
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eacoded in ID Field 504. This control n^essage controls ESs 14 and 95, as indicated by 
Pomter Field 505. The associated Message contams 1,024 bytes, as indicated by Length 
Field 506. ^ 

In an alternate embodiment, the association of control to content may be made in 
Orgamzation Block 8, which may store a pomter to particular control messages along with 
the metadata associated with streams, objects, etc. This may be disadvantageous, however 
in that It may be desirable to protect this association from discovery or tampering by users ' 
Since Control Block 13 will generally have to be protected in any event, storing the 
association in this block may make protection of Organization Block 8 less necessary. 

Control Block 13 implements control over System 1 through Control Lines 14 15 
and 16, which control aspects of Organization Block 8, AV Block 9 and Composite Block 
11, respectively. Each of these Control Lines may allow two-way communication. 

Control Lines 14and 15 are shown as communicating with AV Block Stream Flow 
Controller 18 and with Organization Block Stream Flow Controller 17. These Stream 
Flow Controllers contain functionality controlled by Control Block 13. In the embodiment 
illustrated, the Stream Flow Controllers are shown as the first stage in a two-stage pipeline 
with information being processed by the Stream Flow Controller and then passed on to the 
associated functional block. This allows isolation of the control functionality fVom the 
content manipulation and display functionality of the system, and allows control to be 
added in without altenng the underlying functionality of the blocks. In an alternate 
embodiment, the Stream Flow Controllers might be integrated directly into the associated 
functional blocks. 

stream Flow Controllers 17 and 18 contain Cryptographic Engines 19 and 20, 
respectively. These Cryptographic Engines operate under control of Control Block 13 to 
decrypt and/or cryptographically validate (e.g., perfomi secure hashing, message 
authem.cat.on code, and/or digital signature functions) the encrypted packet streams 
received from Demux 7. Decryption and validation may be selective or optional according 
to the protection requirements for the stream. 

CiTptographic Engines 19 and 20 may be relatively complex, and may, for 
example, include a validation calculator that performs cryptographic hashing, message 
authentication code calculation, and/or other cryptographic validation processes In 
addition, as is described further below, additional types of governance-related processing 
may also be used. In one alternative embodiment, a smgle Stream Flow Controller may be 
used for both Organization Stream 3 and AudioA^ideo Streams 4-5. This may reduce the 
cost of and space used by System 1 . These reductions may be significant, smce System I 
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may contain multiple AV Blocks, each handling a separate Audio or Video Stream in 
parallel. This alternative may, however, impose a latency overhead which may be 
unacceptable in a real-time system. 

If the Stream Flow Controllers are concentrated in a single block, they may be 
mcorporated directly into Demu.x 7, which may handle governance processmg prior to 
routmg streams to the fimctional blocks. Such an embodiment would allow for governed 
decryption or validation of the entirety of Bit Stream 2, which could occur prior to the 
routmg of streams to individual functional blocks. Encryption of the entirety of Bit Stream 
2 (as opposed to individual encryption of individual ESs) might be difficult or impossible 
without incorporating stream controller ftmctionality into Demux 7, since Demux 7 might 
otherwise have no ability to detect or read the header information necessary to route 
streams to ftmctional blocks (.hat header information presumably being encrypted). 

As is noted above, each of the individual streams contained m Bit Stream 2 may be 
mdividually encrypted. An encrypted stream may be identified by a particular indicator in 
the header of the stream, shown in FIG. 2 as Governance Indicator 205. 

When a header is passed by Demux 7 to the appropriate functional block, the stream 
flow controller associated with that block reads the header and determines whether the 
following packets are encrypted or otherwise subject to govemance. If the header indicates 
that no govemance is used, the stream flow controller passes the header and the packets 
through to the functional blocks whh no alteration. Govemance Indicator 205 may be 
designed so that conventionally encoded content (e.g., unprotected MPEG-4 content) ,s 
recogmzed as having no Govemance Indicator and therefore passed through for nomial 
processing. 

If a stream flow controller detects a set govemance indicator, it passes the ES_ID 
associated with that stream and the time stamp associated with the current packets to~ 
Control Block 13 along Control Line 14 or 15. Control Block 13 then uses the ES ID and 
time stamp information to identify which control message(s) are associated with that ES 
Associated messages are then invoked and possibly processed, as may be used for 
governance purposes. 

A simple govemance case is illustrated by FIG. 6, which shows steps which take 
place using the fimctional blocks of FIG. 1. In Step 601, Demux 7 encounters a header 
and detemimes that the header is part of the AV stream. In Step 602, Demux 7 passes the 
header to AV Stream Controller 18. In Step 603, AV Stream Controller 18 reads the 
header and detemiines that the govemance indicator is set, therebv triggenng further 
processmg along Path 604. In Step 605, AV Stream Controller 18 obtams the ES_ID and 

SUBSTITUTE SHEET (RULE 26) 



wo 99/48296 

PCTA;S99/05734 

- 10- 

.5. In step 6,^, C„„«„ Block ,3 loo^ up eSE a.d d«e™i„^ ,h. ES ID is 
™d w„h a pa„ic„,.co„,., message. I„ S,ep 6,,. Co„»o, Block ,3 ,hc ,tac 
^■amp ,Monna..on ,o choose among con»o, messages. ,f tee ,s m„,e one comro, 
message a.30c,a,ed wi* a panieular ES. to S.ep 607, ConTo, Block 1 3 accesses .he 
app,opna,e con.^1 message, and chu,„s a c^,„g„phic key or keys for decvp.io„ and/or 

L- el T .vT ^"^^ " ""^-^ ='^"^">^^ ^«^<=) ^'^^ con J 

L,ne ,5 ,„ AV S«an, Co„«„„er ,8. S.ep 609. AV S.e,m Connoller ,8 uses >he 

c.yp« key as an inpn, .o C,yp,ograph,c Engine .0, which decy,,s a„d/or validates 

e dec^>ed packets are .hen passed .o AV Block , wMch decompresses and proce ses 

^hem m a conventional manner. 

Time s,amp informa.,o„ may be useful when i. is desirable ,o change ,he conwl 
.^.age applicable .o a par.,cular ES. Por example, i, may be use«., .o en!ode d-irerl 
por.,ons of a stream „,.h d.irerem keys, so Ota. an a.tacker breaking one key ,or even a 
num er o keys, w,l, no. be able to use ,he comen.. This can be done by asLcia„ng a 
number of comrol messages with .he same s.ream, with each co„»„l message be,ng valid 
for a pantcular penod. The .ime s.amp i„form«.on would .hen he used to choose wh.ch 
control message ,a„d key, to use a, a particular .,m.. Alten,a.,vely, one con.r„l message 
may be used. bu. wi.h upda.ed mfon„a.,o„ be,ng passed in .hrough .he Con.,ol Stream the 
updates consisting of a new time stamp and a new key. 

In an alternative embodtmem. Control Block 13 may proactively send the 

.o detenntne when a key „,l| be w,ll be needed. This may reduce overall latency 

Control Line 16 Irom FIG. , comes into play once infonnat.on has been passed 
from Organtzanon Block 8 and AV Block 9 .„ Composfe Block 1 1, and the f.mshed work 
.s prepared for rendering through Rendering Device 12. When Composite Block 1 1 sends 
^object to Rendcnng Devtce , 1. Composite Block 1 1 sends a start message to Control 
Block 13. TT's message identtOes the objec. (including any associaledES IDs) and 
spectfies .he Stan .,me of .he display (or oiher rendenng, of .ha, object, ^^en ™ ob,ec, ,s 

longer bemg rendered. Composite Block 1 1 sends an end message to Co„.ol Block 13 
specfymg that rendedng of the object has ended, and the ..me a. which .he ending 
occurred. Muhtple copies of a pa„,cularobjec. may be rendered a, the same fme. FortUs 
reason, star, and stop messages sen. by Composite Block 1 1 may .nclude an assigned 
instance ID, which speciiies which instance of an object is being rendered 
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Control Block 13 may store infonnat.on relatmg to start and stop times of particular 
objects, and/or xnay pass th.s .nformat.on to extental devices (e.g.. External Sender 30) 
thn^ugh Port 21. This .formation allows Control Block 13 to keep track not onlv of winch 
objects have been decrypted, but of which objects have actually been used. This may be 
used, smce System 1 may decrypt, validate, and/or decompress many more objects than are 
actually used. Control Block 13 can also detennme the length of use of objects, and can 
detennme which objects have been used together. Infonnafon of this type may be used for 
sophisticated billing and auditing systems, which are descnbed further below. 

Control Line 16 may also be used to control the operation of Composite Block 1 1 
In pamcular. Control Block 1 3 may store mfonnation specifying when rendering of a 
panicular object is valid, and may keep track of the number of times an object has been 
rendered. If Control Block 13 determines that an object is being rendered illegally (i.e m 

violation of rules controlling rendering) Confml Ri^^i. n . 

5 it.iuciing;, L-onn-ol Block 13 may terminate operation of 

Composite Block 1 1, or may force erasure of the illegal object. 

In an alternate embodiment, the level of control provided by Control Line 16 mav at 
east in part be provided without requinng the presence of that line. Instead, Control Block 
3 may store a hash of the organization infomiation currently valid for Organization Block 
8. This hash may be received through Control Stream 6, or, alternatively, may be 
generated by Control Block 13 based on the information contained m Organization Block 

Control Block 13 may periodically create a hash of the information currentiv 
resident m Organization Block 8, and compare that to the stored hash. A difference mav 
.nd.cate that an unauthorized alteration has been made to the information ,n Orgamzation 
Block 8, thereby potentially allowing a user to render information in a manner violative of 
the rules associated with that information. In such an event. Control Block 13 may take 
appropnate action, including deleting the information currently resident m Organization 
Block 8. 

If System I is designed so that Organization Block 8 controls the use of content bv 
Composite Block 1 1 , so that content cannot be rendered except as is specified by the ' 
organization information. Control Block 13 may be able to control rendering of 
mformation through verifying that the current Organization Block contents match the hash 
Which has been received by Control Block 1 3, thereby eliminating at least one reason for 
the presence of Control Line 16. 

Control Block 13 may also be responsible for securelv validating the origin 
.ntegnty. authenticity, or other properties of received content, through cryptographic 
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validation means such as secure hashing, message authenrication codes, and/or digital 
signatures. 

System 1 may also include an Inter-Rights Point, mdicated as IRP 22. IRP 22 is a 
protected processmg environment (e.g., a PPE) in which rules/controls may be processed 
and which may store sensitive information, such as cryptographic keys. IRP 22 may be 
mcorporated within Control Block 13, or may be a separate module. As is illustrated IRP 
22 may include CPU 23 (which can be any type of processing unit), Cryptographic Engine 
24, Random Number Generator 25, Real Time Clock 26, and Secure Memory 27. In 
particular embodiments, some of these elements may be omitted, and additional 
functionality may be included. 

Governance Rules 

Control messages stored by Control Block 13 may be very complex. FIG. 7 
illustrates the fomi in which the control messages may be stored in Control Block 13 
consisting of Array 717. Column 701 consists of the address at which the control messages 
are stored. Column 702 consists of the identifier for each control message. This function 
may be combined with that of Column 701, by using the location information of Column 
701 as the idem.fier, or by storing the message in a location which corresponds to the 
idemifier. Column 703 consists of the ES_IDs for each stream controlled by the control 
message. Column 704 consists of the message itself Thus, the control message stored at 
location 1 has the ID 15, and controls stream 903. 

In a simple case, the message may include a cryptographic key, used to decrypt the 
contem associated with the stream(s) controlled by the message. This is illustrated bv 
Cryptographic Key 705 from FIG. 7. Cryptographic keys and/or validation values may 
also be included to permit cryptographic validation of the integrity or origin of the stream. 

In a more complex case, the message may include one or more rules designed to 
govern access to or use of governed content. Rules may fall into a number of categories. 

Rules may require that a particular aspect of System I, or a user of System l,be 
venfied prior to decryption or use of the governed coment. For example, System I may 
.nclude System ID 28, which stores a unique identifier for the system. A particular rule 
contained in a control message may specify that a particular stream can only be decrypted 
on a system m which System ID 28 contains a particular value. This is illustrated at row 2 
m FIG. 7, in which the message is shown as consisting of a rule and commands. The rule 
may be implicit, and therefore may not be stored explicitly in the table (e.g. the table may 
store only the rule, the rule - specific functions (commands) invoked by the rule, or only 
the functions). 
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.his ce. When S^>r„ Co„„„„„ , s e„c„„„„, , „^ 

Mjage20wh,chgover„ss«»,203.. Conm,, Message 20 ,„c,ud=s R„,e 706 which 

iiysiem IL) 708 may have been received bv Svitrm i .i.i, 

ivcu oy system 1, either as part of Control Me<;saa^ 9n 
oraspanofanoUierconnolmessasele, r . ■ ™mi Message 20, 

conld,h.n,.f '""*°*=-8-<^>"«™IM=='i''8e'). which Control Message 20 

conM ihen reference ,„ order ,„ obtain access .o ,he Au,h„„.ed Sysren, ID. Such a cl 

™gh.ex.s,, forexa^pie. ifa cab,, subscriber had pre-regrstered fo,apre„,u.„ sh wL 
caWe sysie™ .,gh, reco^^e iha. registrar, ahd aurhonzc the user ,o view" e sZw^v 
s«,d,„g to the user an ID corresponding to the Syscen, ID ' ' 

o.a,„z:trz:::re'~'"^"'"' -------- 

Systetn ID ^O. spec, Zb 7:Tr^^^^^^^ 'T"' 

r,^,„ u „ "^"""""""'"''•'^ match. Commands 707 release 

Cryptogr^phtc Key 709 to Stream Con.ol,cr , 8, which uses Cryptog^phtc Key 70 to 
^actypt , e stream co^pondmg to ES.ID 2031. If the »o no^a. 
Commands 707 fail to release C^tographic Key 70,, so that Stream Control '.s 
unable to decrypt the stream. >-ontroiler I s is 

In order to carry out these fimcttons, m one embodiment. Control Bl«.k ,3 
«.ble o executtng a^y of the con^ands which may be .ncluded or invoked by any of ^e 

message and IDs of any governed ESs). 

c mpletely protected by a banrer which resists tampenng and obseryat.on As ,s dlnbed 
^ve, theprocessing un,. secure m^ory, and vanous other govemanccrela^ e™ 

example a clr ~ "-^'^ "P™-- °- 

exampi , a control message may require .hat information from Sys,em 1 no. only be 

accessed and compared to exoecieH i„f„„ . u 

P 0 to expected information, but stored for (iiture use. For examole a 
control message might allow decryption of a Stream, but only after Sys.em ID JsT K 
downloaded .o and s.ored in Control Bloc. ,3. This would L a 
Che* the stored System ID agamst System ID 28 on a regular ba.is, or perhZr; 
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attempted re-viewing of a particular Stream, thereby allowing the control message to msure 
that the Stream is only played on a single System. 

Control Block 13 may also obtain mformation dynamically. For example. System 1 
may mclude User Interface 29, which can include any type of user input functionality (e g 
hardware buttons, information displayed on a video screen, etc.) A particular rule from a ' 
control message may require that the user enter information poor to allowing decryption or 
use of a stream. That infomiation may, for example, be a password, which the Rule can 
then check agamst a stored password to msure that the particular user is authorized to 
render the stream. 

Information obtained from the user might be more complicated. For example a 
mle might require that the user input paymem or personal information pnor to allowing 
release of a cryptographic key. Payment infomiation could, for example, constitute a credit 
card or debit card number. Personal information could include the user's name, age, 
address, email address, phone number, etc. Entered information could then be sent 'through 
Port 21 to External Server 30 for venfication. Following receipt of a verification message 
from Extemal Server 30, the Rule could then authorize release of a cryptographic key 
Alternatively, Control Block 13 may be designed to operate in an "off-line" mode, storing 
the mformation pending later hookup to an extemal device (or network). In such a case, 
Control Block 13 might require that a comiection be made at periodic intervals, or might 
limit the number of authorizations which may be obtained pending the establishment of an 
external connection. 

In a somewhat more complex scenario, a confrol message may include conditional 
rules. One particular example ,s illustrated by row 4 of the table shown m FIG. 7, in which 
Control Message 700 is shown as controlling streams 49-53. Control Message 700 further 
consists of Rule 710, Commands 71 1 and Ciyptographic Keys 712-716. There could, of 
course, be a number of additional cryptographic keys stored with the message. 

In this case. Rule 710 specifies that a user who agrees to pay a certain amount (or 
provide a certain amount of information) may view Stream 49, but all other users are 
required to view Stream 50, or a combination of Streams 49 and 50. In this case, Stream 
49 may represent a movie or television program, while Stream 50 represents 
advertisements. In one embodimem, different portions of Sfream 49 may be decrypted 
with different keys so that, for example, a first portion is decrypted with Key 712, a second 
portion is decrypted with Key 713, a third portion is decrypted with Key 714, and so on. 
Rule 710 may include all keys used to decrypt the entirety of Stream 49. When the user 
initially attempts to access the video encoded in Stream 49, Rule 710 could put up a 
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message asking if the user would prefer to u^e n:.v Cr.. ■ 

, , P'"^'^""^^ pay for view mode or advertising mode If 

md,ca„„g .ha, a diffe™ key is „«ded ,0 te^, f„„„„,„g 3=, of pacL Upon 

d-nn,. me fol,ow,„g pactes. and so on. Rule 7,0 could addi>iona„y release 
C^ptographic Key 7,6. co,respondi„g ,0 O^amzaUon S»ean, 52, »hich co^sponds ,0 
video without advertisements. "^ponosto 

If, on the other hand, the user had chosen the adverttsing mode, R„,e 7, 0 could 

Rule 71 0 could taher release Ctyptographic Key 7,5 ,0 Orgartzation B,«:k 8 
Cryptopaphtc Key 715 matches Orgam^tton Stream 5,. Organization Stream 51 
references the v.deo trom Stream 49, hut a,so references adverttsemcnts m,m Stream 50 
Rule 710 would refi.se to release Ctyptographtc Key 7,6, which eotrespo^ls to 
Organtzatton Stt^am 52, which cotresponds to the vtdeo without advenisemenu 

operation, Con.ro, B,ock , 3 cou,d momtor ,„fon.at.on fron, Composhe B,ock 
n overCo„tro>L,„e .6. That mfomtatton coutd tnCude the identtty of each ohject 
ae^aUy rendered, as we,, as a stan a„l stop t.me for the .„den„g. Con.o, B,oc. , 3 
cou,d us, ,h,s mformatton to detennine that an adven.scmen, had actua„y heen rendered 

r T;~''''^'^'"'-*=-^''-°^'''---PO".«nofv,de: 
tan,S«am49. Th,s feedback ,oop ahows Contro, Bloc. ,3 to he ceruin that the 

ven,se™ents are not only being dec^ted, bu, are also betng displayed. TWs may be 
-essary because Composite B,». , 1 may be relat.vely unptotected, thetebv allowm an 
unscrupulous user to remove advenisements before viewing " 

A vanety of addihoual relatively complex scenanos are possible. For example 
™ s front Control B,oc. , 3 could customize the programming for a particular geo^^hic 
caiion or a p„,cu,ar type of viewer, by using infomtation on the iocation or the vL 

7,8 Jr"" ^"""^ " ' °f R«'= '19 may specity Budget 

7,8, Which may include infonnation relating ,„ the nunther of uses avai,ab,e to the user 
Ote amount of money the user has to spend, etc. ,„ operation, Ru,e 7,9 „ay require thl; 
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Budget 7 1 8 be securely stored and decremented each time a budgeted activity occurs (e g 
each tmie the associated work is played). Once the budget reaches zero. Rule 719 may 
specify that the work may no longer be played, or may display a message to the user 
mdicatrng that the user may obtain additional budget by, for example, entering a credit card 
number or password, or contacting an external server. 

In another example, a rule may control the ability of a user to copy a work to 
another device. The rule may, for example, specify that the user is audionzed to use the 
governed work on more than one device, but with only one use being valid at any tmie 
The rule may specify that an indication be securely stored regarding whether the user has 
"checked out" the work. If the user copies the work to another device (e.g., through Port 
21), the rule may require that the work only be transmitted in encrypted form, and that the 
relevant control messages be transmitted along with it. The rule can further require that an 
mdicator be securely set, and that the mdicator be checked each time the user attempts to 
use or copy the work. If the indicator is set, the rule might require that the work not be 
decrypted or used, smce the user only has the right to use the work on one device at a time 
and the mdicator establishes that the work is currently "checked out" to another device and 
has not been checked back in. 

The receiving device may include the same type of indicator, and may allow the 
user to use the work only as long as the indicator is not set. If the user desires to use the 
work on the ong.nal device, the two devices may communicate, with the indicator being set 
m the second and reset in the first. This allows the work to be stored in two locations, but 

only used in one. 

In another embodiment, the same result may be reached by copying the relevant 
control message from one device to the other, then erasing it from the original device. 
Because the control message includes keys used for decryption, this would insure that the 
work could only be used in one device at a time. 

In one embodimem, this technique may be used to communicate digital media files 
(e.g., music, video, etc.) from a personal computer to a consumer electronics device 
without allowing the user to make muhipie choices for simultaneous use. Thus, a larger 
more sophisticated device (e.g., a personal computer), could download a file, then "check 
out" the file to a portable device lackmg certain fiinctions present m the personal computer 
(e.g., a hand-held music player). 

Rules may also be used to specify that an initial user may transfer the file to another 
user, but only by giving up control over the file. Such rules could operate similarly to the 



SUBSTITUTE SHEET (RULE 26) 



wo 99/48296 

PCT/US99/05734 

- 17- 

techmque descnbed above for transfemng a file from one dev.ce to another, or could 
require that the onginal file be entirely erased from the onginai device after the transfer 

Rules in Control Block 13 may be added or updated through at least two channels 
New niles may be Obtained through Confrol Stream 6. If a control message contains an ' 
.demifier corresponding to a control message already present m Control Block 13 that 
control message (including contained rules) may overwnte the original control message A 
new nxle may, for example, be idem.cal to an existtng rule, but with a new time stamp and 
new keys, thereby allowing decryption of a stream which had been encrypted with multiple 
keys. System 1 may be designed so that certain rules may not be overwritable. This may 
be enforced by designating certam positions m Array 71 7 as non-overwritable. or by 
providing a flag or other indicator to show that a particular nile cannot be overwritten or 
al^red. This would allow for certain types of superdistnbution models, including allowing 
a downstream distnbutor to add rules without allowing the downstream disfributor to 
remove or alter the rules added by upstream distributors. 

In addition, new rules may be encoded into Orgamzation Sfream 3, Audio Sfream 4 
or Video Stream 5, in the form of a watermark or steganographic encoding 

New rules may also be obtained through Port 21. Port 21 may connect to an 
external device (e.g., a smart card, portable memory, etc.) or may connect to an externa, 
network (e.g.. External Sender 30). Rules may be obtained through Port 21 either m an ad 
hoc manner, or as a result of requests sent by Control Block 13. For example. Control 
Message 1 4 (FIG. 7, row 6) may include a rule specifying that a new rule be downloaded 
from a panicular URL, and used to govern Stream 1201. 

Control messages, including rules, may be encoded usmg secure transmission 
formats such as DigiBoxes. A DigiBox is a secure container means for delivering a set of 
business rules, coment description information, content decryption infonnation and/or 
content validation mformation. One or more DigiBoxes can be placed into the headers of 
the media content or into data streams within the media. 

FIG. 12 Illustrates one embodimem of the DigiBox format and the manner in which 
that format is incorporated into a control message. Control Message 1201 is made up of 
Confrol Message Header 1202 and Confrol Message Contents 1203. As is described 
e sewhere. Control Message Header 1202 may include information used by Demux 7 (FIG 
1) to appropriately route the message to Confrol Block 13. 

Control Message Coments 1203 of Confrol Message 1201 consists of DigiBox 
1204, and may also include additional information. DigiBox 1204 consists of DigiBox 
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.207 may include various ,ypes of dau, ,„cWing ES_,D , 208. Co„.ographic Key 120, 
and V^,dau„n Da,a ,210. Dau ,207 „ay also ,nc,ude c,yp,o^phic infonna.,on such as 
specficauon of U,e e™=,yp,ion algoriUnn. chaining m«ie3 used wid, ,he algorithn, keys 
and mitialization vectors used by ihe deciyplion and chaining. 

';^"^'-'™™'<>-co«au,edw,Un„Da,a,207a,esimda„ocr^,ographickeys 
m . a, they cons„n.te input to the ong.a, encryption process and therefore are necessary ' 
for decryption ta one we„-know„ prior an embcdinent, the initializatron vectors may be 
genera^ by stariing with a base i™„a,i.atio„ vector ,a 64 hit tandom „„n,ber, and xofiug 
m the frame number or start time for the content item. 

Validauon Data ,2,0 conumed wtthin Data ,207 may inCude cryptographic has or 
authenttcatron values, ctyptographrc keys forca,cu,ating keyed authenticafon values ,e g 
message authenttcatron codes), digttal signatures, and/or pubhc key ceriificates used in ' 
validalmg digital certificates. 

Tbus. the DigrBox may mcotporate d,e information described above as part of the 
control message, includmg the rules, the stream ,D and the ctyptographrc keys »d values 

In an altemattve embodimen,. DigrBox Header ,205 may be designed so that ,t can 
read by D^ux 7 and routed to Conn., Block 13. to such an embodtment. DrgiBox 
1204 would rtself constitute the entrrety of the control message, thus obviatm. the need to 
nest DigrBox 1204 within Conm)l Message 1201. 

incluH p°?' IT"^ """"" '""""y ^= -^-W""- This nay 

nclude Rules ,206. Data 1207. and possibly some or al, of Header ,205. System I may be 
designed so that a DigiBox may only be dec^ted (opened) ,n a protected en.ronment 
such as 0^ 22. In an alternate embodiment. Control Block ,3 may drrectly incon^rate the 
tactionahty of IRP 22. so that the DigiBox may be opened in Control Block 13 Ibou, 
the necessity of routing the DigiBox to DIP 22 for processing, ta one embodiment the 
cwcgraphic key used to decrypt DigiBox ,204 may be stored in ffiP 22 (or Control 
Block ,3), so that the DigiBox can only be opened in that protected enviromnen, 

R"l=^1206arera,esgovermngaccesstooruseofDigiBoxDalal207 to one 

Key 1209 can only be accessed and used through compliance with Rules 1206. however 
Rules , 206 in fact indirectly control dte governed streams, stnce dtose streams can only be 
ectypted through use of the key. which can only be obtained in compliance with the niles 
to a„oto„ embodiment. Data ,207 may include additional t.,es, which may be extracted 
from the DigiBox and stored in a table such as Array 7,7 of FIG. 7. 
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The rules governing acces. to or use of a DigiBox may accompany the DigxBox. (as 
sho.-nmFIG. 12) or n,ay be separately transmitted, in which event Rules 1206wouid 
contam a pointer or reference to the rules used to access Data 1207. Upon receipt of a 
D.g:Box, Control Block 13 may rece.ve t.les separately through Control Stream 6, or may 
request and receive rules through Port 21. 

Pipelined Implementation 

Ot^e potential drawback to the system illustrated in FIG. 1 consists of the fact that 
the system mtroduces complexity and feedback into a p.pelmed system designed to render 
content m real time. The rendering p.peline generally consists of Demux 7, Organisation 
Block 8 and AV Block 9, Composite Block .1 and Rendering Devtce 12. Because content 
.s received in a streamed fashion, and must be rendered in real t.me, pipelined processing 
must occur in a highly efficient manner, ur,der tight time constraints. A failure to process 
Within the time available may mean that output to Rendenng Device 12 may be tmerrupted 
or that incoming Bit Stream 2 may overflow available buffers, thereby causing the loss of ' 
some portion of the incoming data. 

An alternative embodiment of System 1 is designed to address these problems 
although at a possible cost in the ability to use standard system components and a possible 
cost in overall system secunty. This alternative embodiment ,s illustrated in FIG 1 1 
which shows System 1101. 

System 1 101 is s.m.lar to System 1 from FIG. 1 ,n many respects. It receives Bit 
Stream 1 102, which consists of Organization Stream 1 103. Audio Stream 1 104 Video 
Stream 1 105 and Control Stream 1 ,06. These streams are received by Demux 1 107, which 
passes Orgamzation Stream 11 03 to Organization Block and passes Audio Stream 1 104 
and Video Stream 1 105 to AV Block 1 109. Organization Block 1 108 and AV Block 1 ,09 
operate similarly to the.r counterparts ,n FIG. 1 , and pass information to Composite Block 
1110. which organizes the information into a cohercm whole and passes it to Rendenng 
Device nil. Streams sent to Organization Block 1 108 are decrypted and/or validated bv 
Stream Flow Controller 1 1 12, and streams sent to AV Block 1 109 are decrypted and/or ' 
validated by Stream Flow Controller 1113. 

System 1101 differs from System I, however, in that confrol and feedback are 
distnbuted. and integrated directly into the processing and rendenng pipeline. System 
1101 thus lacks a separate control block, and also lacks a feedback path back from the 
Composite Block 1110. 

In System 1 101, control is exercised directly at Orgamzation Block 1108 and AV 
Block 1 109. As in System 1, cryptographic keys are received through Control Stream 1 106 

SUBSTITUTE SHEET (RULE 26) 



wo 99/48296 



PCT/US99/05734 



20 



( m an alterative embodiment, the keys could be mcorporated directly into header or other 
.nfoxmanon m Organization Stream 1 103 or Audio/Video Streams 1 104 and 1105) Those 
keys are mcluded in a data format which mcludes mformahon regardmg the stream' type of 
the encrypted content and, if multiple stream types are possible, an identifier for the 

particular controlled stream. 

When Demux 1 107 encounters a key m Control Stream 1 106, k reads the 
.nformafon re.atmg to the stream type, and routes the key to the appropriate stream flow 
controller. IfDemux HOT encounters a key designated for decryption or validation of 
Organization Stream 1 103. for example, it routes that key to Stream Flow Controller 1,12 

Stream Flow Controller 1112 stores received keys in Storage Location 1114 
Storage Location 1 1 14 stores the keys and also stores an indicator of the controlled stream 

S^eam Flow Controller 1112 mcludes Cryptographrc Engine 1 1 ,5, which uses the 
received keys to decrypt and/or validate encrypted and/or protected portions of 
Organization Stream 1 103, The keys may themselves be received in an enctypted manner 
m order to provide some degree Of secunty. In such a case. Stream Flow Controller may ' 
use a vanety of techniques to decrypt the key. including usmg stored information as a key 
or as a key seed. That stored information could, for example, constitute a "meta-key" 
provided earlier through Bit Stream 1 102 or through a separate port. 

Stream Flow Controller 1113, associated with AV Block 1 109. contains a 
corresponding Storage Location 1 1 1 6 and Cryptographic Engme 1 1 1 7, and operates ,n a 
manner Similar to the operation described for Stream Flow Controller 1 1 12 

This implementation avoids the latency penalty which may be inherent m the 
necessity for communication between stream flow controllers and a separate control block 

This alternate implemenution may also eliminate the feedback chamiel from the 
composite block (FIG.l, Control Line 16). Tins feedback channel may be used m order to 
msure that the content being passed from Composite Block 1 1 to Rendering Device 12 is 
content that has been authonzed for rendenng. In the alternate embodiment shown m 
FIG.l 1. this feedback chamiel does not exist. Instead, this implementation relies on the 
fact that Composite Block 1 1 10 depends upon mformation from Organization Block 1 108 
to detemiine the exact structure of the information being sent to Rendering Dev.ce 1111 
Composite Block 1110 camiot composite information in a mamier contrary to the 
orgamzation dictated by Organization Block 11 08. 

In one embodiment, this control by Organization Block 1108 may be sufficient to 
obviate the need for any feedback, smce Organization Block 1 108 may be designed so that 
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may d^,^,d so ,ha. „ o„,y dcc^. „, v,u,,,, 
Stored in Storage Location 1114. 

In such an embodiment, security mav h/. fi,«Ko ... 
, , ' ^^'-""^ may be fimher mcreased bv mcorporating Secure 

Memory , , „ o.„ B,ocU , OS. Se.^ Me„„^ , „ s „,y L a i y oT 

u« m M ,„ orgamza.™ Block M^ory „ ,9. Orga„,za«o„ Block , ,08 ™y be „sed ,0 
er,o ,ca^,y compa. ,Ke o.ga„,.,io„ ,.e s,„^ i„ Ma.„ Orga™.Ho„ Block Memo^ 
ni9,o,he»ees,oredmSec„reMen,o,y ,„8. If a discrepancy i. spo»ed, , hi. ™ay 

U , .hereby poss,.,y allowing for .he re„de„„g of eo„.e„, i„ yio,a.io„ of apphcahle 
™les. Under such c,rc™s«ces, Orga„iza„o„ Block 1,08 may be used .o ,ake pro.ecive 
™, ,„c lu .„g replacng .he co„.c„,s of M„„ Orga„„,,o„ Block Memory | 
the contents of Secure Memory 1118. y » * ' > wim 

MPEG-4 ImplemeDtation 

TI« generic syslen, descnbed above may be embodied in an MPEG-4 system as 
.Uus»a.ed in FIG. 8. which shows MPEG-4 Syslem 801. 

MPEG-4 Sys,em 801 accept MPEG-4 Bi, Stream 802 as .npu.. MPEG-4 Bi. 
Stteam 802 .ncludes BIFS Strean, 803, OD Strean, 804, Aud,o Stream 805. Vdeo Stream 
6 and I^MP Strean, 80. These streams are passed to Oetnux 808, „h,ch exammer 

"r;::::;: " ^^"-^ •» -o 8., ob 8,1 or 

n>MPSys.em812rece,vesIPMP,nessagesthroughIPMPStream807 Those 

^.at^ O.MP message T.e D-MP message may include control infonnation. which 
may mclude a coT.tograph.c key, validation ,„fonna.,„„, and/or may tnclude complex 
governance rales, as are described above. 

Street Con.ro,le. 813, 8,4 and 815 ac. ,0 decrypt, validate, and/or govem streams 
passed to BIFS 809, AVO 810 and OD 8 1 1 . respectively 

OD 81 1 holds object descnptors. which contain metadata deserting particular 
objects. Thts metadau i™=ludes an tdentifter of the particular Elemcntaty Stream or 
streams which tnclude the object, and may also include a pointer to a panicular IPMP 

ZZ T r™ ses 
»d parttcular objects or streams may be stored m a table or other for™ withm IPMP 
bystem 812. 
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1, irmi' :iystem 8 12 confirms that the current nw^ 
.he a„„„n.e. .ec ^e,ve. b,FS S.e™ 803. :pZ1^17'T " . " 

the proper content is beina r^nw ^ ^ ^ ^^"^"^ that 

P °"'^"^"^b«'ng rendered, even without recemng feedback directly from 
Composite and Render X91 ■n,;o u "'^'-R- uireciiy rrom 

-urn is.enaer 82 1 . This may be necessary since RFK*; sno ^ 
with Port 822 which m.v .II communicate 

which may allow a user to insert infonnation into BIFS 809 iher.K 
creating a possibility that a user could in..rt u ^ 

unauthonzed acces^to content " ""^^ ^^^^ '^^^^ 

.ay senlTnTZ" ^^^^^ ™ " ~ n, it 

send this :;:^:::::tt:''' ^^^^^'^ - - 

goyems that object or strer i s "^^^ ''' ' ^ ''''' "^^^^^ 

re.uestdec^ton y 1 Zn ^ ^^^^^ ^PMP message ID to 

ODSl, : IPMP System 812. Altematiyely 

OD 8 1 1 can pass the IPMP ID to IPMP System S n Alternatively, 

appropnate stream controller. ' '''^^ '"""^^ ^''^ ^''^ 

connected toadevic^lreC^^ 

a aevice or memory (e.g., a smart card, a DVD disk etr ^ «r . 

network (e.g.. the Internet). An IPMP message mav . 

obtainable through Port 8 12 , ^'^^ ^ P^'"*- »° '"forrnation 

tnrough Port 812, such as a URL, address on a DVD disk etc TTiatfrni 
contain specific controls needed by the IPMP ^« ""^^ 

information, such as for examt,le r " "'"""^ ^^^"'^^'^ 

n>MP System m ' " '"'^^^ P^-'- 

contained in OD Stream Rf)d nn c*^ «^ upaaies 

Header 902, wluch .dentite ,h. following packets as pan of ,he OD 
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stream, and indicates the number of packets. OD Message 901 further consists of Message 
903, which includes a series of Pointers 904 and associated Metadata 905. Each Pointer 
904 Identifies a particular Elementary Stream, and the associated metadata ,s applicable to 
that stream. Finally, OD Message 901 may contain an IPMP Pointer 906, which identifies 
a particular IPMP message. 

In aggregate, the information contained m OD Message 901 constitutes an object 
descnptor, since it identifies and describes each elementary stream which makes up the 
object, and identifies the IPMP message which governs the object. OD Message 901 may 
be stored in OD 81 1, along with other messages, each constituting an object descnptor. 

Object descriptors stored in OD 81 1 may be updated through OD Stream 804 
which may pass through a new object descnptor corresponding to the same object The 
new object descnptor then overwntes the existing object descriptor. This mechanism may 
be used to change the IPMP message which controls a particular object, by usmg a new 
object descnptor which is identical to the existmg object descriptor, with the exception of 
the IPMP pointer. 

OD Stream 804 can also carry IPMP_DescriptorUpdate messages. Each such 
message may have the same fomiat as IPMP messages carried on the IPMP stream, 
mcluding an IPMP ED and an IPMP message. 

IPMP_DescriptorUpdate messages may be stored in a table or an-ay in OD 81 1 or 
may be passed to IPMP System 812, where they may overwrite existing stored IPMP ' 
messages, or may add to the stored messages. 

Since IPMP infomiation may be separately conveyed through the OD stream or the 
IPMP stream, MPEG-4 System 801 may be designed so that it only accepts information 
through one or the other of these channels. 

In another embodiment, the existence of the two chamiels may be used to allow 
muhi-stage distnbut.on, with governance added at later stages, but with no nsk that later 
alterations may override governance added at an earlier stage. 

Such a system is illustrated in FIG. 10. In this Figure, IPMP System 812 includes 
IPMP Table 1002, which has slots for 256 IPMP messages. This table stores the IPMP ID 
.mpl.c.tly, as the location at which the mfomiation is stored, shown in Column 1003 The 
rPMP message associated with IPMP_ID 4, for example, is stored at slot 4 of IPMP Table 
1002. 

Each location in IPMP Table 1002 includes Valid Indicator 1004 and Source 
Indicator 1005. Vahd Indicator 1 004 is set for a particular location when anlPMP message 
IS stored at that location. This allows IPMP System 812 to identify slots which are 
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unfilled, which otherw,se might be difficult, since at start-up the slots may be filled with 
random mformat^on. This also allows IPMP System 812 to identify messages which are no 
longer vaUd and which may be replaced. Valid Indicator 1 004 may store time stamp 
mfonnation for the penod dunng which the message is valid with IPMP System 812 
detenmmng validity by checlong the stored time stamp mformation against the currently 
valid time. 

Source Indicator 1005 is set based on whether the associated IPMP message was 
received from IPMP Stream 807 or fi-om OD Stream 804. 

These indicators allow IPMP System 812 to establish a hierarchy of messages and 
to control the mamier in which messages are added and updated. IPMP System 812 may 
be designed to evaluate the indicators for a particular location once a message is received 
corresponding to that location. If the valid indicator is set to invalid, IPMP System 812 
may be designed to automatically wnte the IPMP message into that slot. If the valid 
indicator is set to valid, IPMP System 812 may then be designed to check the source 
indicator. If the source mdicator mdicates that the associated message was received 
through OD Stream 804, IPMP System 1812 may be designed to over^vrite the existing 
message with the new message. If, however, the source indicator indicates that the 
associated message was received through IPMP Stream 807, IPMP System 812 may be 
designed to check the source of the new message. That check may be accomplished by 
examining the header associated with the new message, to determine if the new message 
was part of OD Stream 804 or part of IPMP Stream 807. Alternatively, IPMP System 812 
may denve this information by determining whether the message was received directly 
fi-om Demux 808 or through OD 811 . 

If the new message came through IPMP Stream 807, IPMP System 812 may be 
designed to store the new message in Table 1002, overwriting the existing message. If the 
new message came through OD Stream 804, on the other hand, IPMP System 812 may be 
designed to reject the new message. 

This message hierarchy can be used to allow for a hierarchy of control. A studio 
for example, may encode a movie in MPEG-4 format. The studio may store IPMP 
messages m the IPMP stream. Those messages may include a requirement that IPMP 
System 812 require that a trailer for another movie from the same studio be displayed prior 
to the display of the feature movie. IPMP System 812 could be used to monitor the 
beginning and end of rendering of the trailer (using feedback through Control Line 819) to 
ensure that the emire trailer plays, and that the user does not fast-forward through it 
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c^U cha^eL The ™v,e s,„d,o could provide ,he distributor „id. a key enabling U,e 

,n de 'T" Tr^'"'" f-xanrtle 
ser p.d for p,en„un, viewrng, dec^ ^e movie if pr™i„„, v,ewing has been paid 

by he s.nd,„ ,0 a nval „ov,e being ^o,., by ,he cable channel^ The smdio's mles 

sp^ily .he ,^es or„e» ™les „h,ch would be allowed .hrough ,he OD s a! 
(hereby providing the sn,dio a high degree of con»^l. 

TOs same mechamsm could be used to allow superdistribudon of comeM possibly 

tte:::l:::^"'°'"^^"^'''°"'"'--'''-»•*-— -.nterfae:^ ; 

be insertion of message, into the OD s^eam. A user might, for example, inseri a message 

". wed. T e user could then provtde the movie ,o another user (or distribute it through a 
medium whereby copying ,s uncontrolled, such as the Internet,, and s.ll receive payi^^^n, 
Because the user. ..les could no. ovemile the suidio. niles. however, the smdio coTh 
cenam that its niles would be observed. Those might include rules speci^ng the Z c 
rules a user would be allowed to add (e.g., limiting the pnee for rcdiLbuTon,. 

svsi .V ""^'^^ '"'"•^ ' type of ffMP 

^.em. Which may be incompadble with H-MP systems that may be design": into ol 

t^fo™ . r . ^'^^^ " specify 

.he format of the mfonnation cont^ncd ,„ the ^MP s.eam. thereby allowing different 

content providers to encode informattonmdiffetmg manners 

Which dTf"" " '''"^ ""^ "^'^^ - -ironmen, .„ 

w^ch diffenng ™p ro,r..s cxis.. That system may sc. the .PMP s.cam for headets 
4a, compattble with IPMP System 812. Al, other headers ,a„d associated packets) 

^e IPMP message in multiple fo^ats, without any concern tha, ene„„n.en„g „ 
unfamihar forma, would cause an IPMP system to fail. In par^cular. IPMP heaL can 
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incorporate an IPMP System Type Identifier Thn.. a 

r^ntroi .1. laenntier. Those identifiers could be assigned bv a 

r •» '^^^ n-MP Sys,™ IdenHfler. discarding a„ 

Keade„, ,nc,ud,„g heade,. ,„co,p„™„8 al,en,a.c IPMP System Id^ifiers !, ■ 
by the IPMP syaem. Identifiers also recognized 

order dte™ ftom „„3. to tet preferred, by .nCudntg the most pteferred fonnat fa. the 
second ™„st preferred fonnat second, and so on. Since IPMP Svstetn 80, ,7. ! 

n^o.patihIefor.na,,tfinds,.,so,der.n.„.MPS.rs::::r^^^^^^^^^^ 
IPMP system chose the fonnat .„st desi,«, hy the content provider 

Even ,f different IPMP formats are nsed, content »il, probably be encoded tand 
encrypted) nsi„, , single algonthn,, ^^^^^ of co iTwo ^ 

.pose a stgnifican. bandwidth bt^den Thns, on^narily ,, ^„ be necessa^nl, to 

use the DES algonthm in output feedback mode. 

This method of screening IPMP header.: ^r>A i^^u- 

a]<?oh^„c.H. "^^''^"'^'^'o^king onto a particular format mav 

aJso be used to customize an MPEr,-4 hiKif,.. c ^ . •"«"indy 

particular MPEr-4 . c ' capabilities of a 

particular MPEG-4 system. Systems capable of rendering MPEG-4 content m.v 
considerable range of functionality from hi^ enH H u ' 
ro„ high-end home theaters to handheld devices 

Governance opttons snttable for one type of system may be trrelevan, to other sy Is 

.hro../;::rT'"""''" -^^ 

inrougn f^ort 820, whereas a second MPFr,..! c.,o+^ /r 

li^c device, may lac. such a co Jctr!l~7""'': ' ^ 

The ™,e conid .hen send the infonnatton throngh a pot, to the Interne,, to a UlTsp c fied 
^ .be ™,e. A si.e a, that URt conid then evaluate Ure ttser i„f„,m.tio , andZnId 
adverttsements targeted to the particular user 

nosenserairrrh^rotT^r^---^^^^ 

evice which IS not necessanly connected to the Internet. It would make no 
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extemaj URL or download the advertisemenf.: r„ c, u 
prefer to requ.re that the user watchp^sllVd^^^^^ 

stream. Preselected ads contained in the original MPEG-4 bit 

Header information in the IPMP stream rr,..iH k„ 
^tr^.r^f • . ''e used to customize an MPEG-4 hit 

stream for particular devices. As with the IPMP Svstem Tv. ^ 
infi,,™,* ... :>ystem Type information, IPMP Header 

information could include MPEG-4 Sv^tem T,^ -ru 

wi* pan... ...^.s ™ C"^^^^ ' " 

, ^ y maps. 1 hus, the presence of a bit at Do<?itinn ? 

:..:pr::;:::— ^ 

-.~a„MPeO-.™.oU 

particular header, and would download the IPMP n..cc u « 'ock on to a 

-n,«c» messages characterized by that header 

Those messages would prompt the user fnr i.r . 

the URL .nH w M u • 'nfomiation, would provide that mfonnation to 

he URL, and would authonze decryption and rendering of the movie, with the 
advertisements mserted at the appropriate spot 

~n::™^^^^^ 

u uuwnioaa uie rules associated w th that heaHpr Thr..^ i i 
no. provide any „p,i„„ „ ,he ^„ ^ ""Sl" 

would also »,.if, * ""^ ''^^Wion of the content, but 

would al o speciy dec^Ttton of an additional ES f^o™ the MPEG.4 stream Tlra, 

-.ow.„.d_.dre.::::ere~^^^ 
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would never be loaded, since .hose n,le. would be contained wihin messages ,de„,iiled by 
.he wrong ^ of heeler. Tl,e advenisemen. ES would .he„fo. never he decyp.ed and 
would be Ignored by the MPEG-4 device. 

FIG. 21 ill„s.ra,es one manner in which a pro.ec.ed MPEG-l flie may be c„a.ed 
In h,s fig... Crea,eBox 2,01 represents a DigiBox crea.,o„ u.i,i.y, whtch accept keys' 
and rules, taone embodiment CrealeBox 2101 may pass tee keys and rules .„ IRP 2102 
and reccve DigtBox 2,03 from KP 2102. In an„*er embodiment IRP 2,02 may he 
.neorp„ra.ed in.o CreareBox 2,0,, which accep.s keys and rules and ou,u,s DigiBox 

MfBox 2103 conlains governance rules, inidaliza.ionvec.ors and keys. DigiBox 
2,03 ,s passed from C,ea.eBox 2,0, .0 B,f Encoder 2, 04. Bif E„c«ler 2104 may be 
«o„a, wi.h .he excep,ion tha. i. is designed to accept and process DigiBoxes such 
as D. . Box 2,03. Bif Encoder 2,04 a,so acccpis a .txt file contain.ng a scene graph a™, 
initial object descriptor commands. 

Bif Encoder 2104 outputs a .bif file, contaming the scene graph stream (in 
compressed bina^ form) and a .od file, containing the imtia, object descnptor cormnands 
me object descnptor stteam, and DigiBox 2 ,03. 

Bif Encoder 2 104 passes the .bif file and the od file to Mux 2105. Mux 2,05 also 
accepts compressed audio and video files, as well as a scr file tha. co„.atns d,e strean, ' 
descnptton. Mux 2105 creates IPMP s»eams, descdp.o,s and tnessages, e„c,yp.s ,he 
cc,ntem s.reams. .n.erleaves the received streams, and outputs Protected MPEG-4 Content 

2 06. conststtng of Initial Object Descnptor 2,07 and Enctypted Content 2,08. I„,„a, 
Object Descnptor 2,07 contains D,g,Box 2,03, as w.„ as other .nfo^tatton. Encrypted 

' "™ ""^ ' '''' ^-">- " ""J" ^-"P- 

stream. IPMP streams, and cnctypted content strean,s. 

'f™StBox2103conta,nsallkeysandn.lcsnecessarytorcnderall„ftheco„.ent ,, 
may be unnecessary for Mux 2105 to create any IPMP streat^s. If additional keys or ruUs 
may be necessary for at ,e«.t a pon.on of the content. Mux 2,05 may ineotporate those 
mies and keys ,nto one or more additional DtgiBoxes, and inc„n,ora.e those DigtBoxes 
clher m the IPMP stream or in the OD update stream. 

'=■0 22 illustrates one manner in which connt,, may be incorporated into an 
ex.„ng MPEG-, stream. M this figure, Unpro.ec.ed MPEG-4 Co„.en. File 2201 includes 
Uuual Object Descnptor 2202 and Con.en, 2203. The content may include a scene 
descnption s„e^ („r BIF s.eam,. an ohjec. descrtptor stream, a video stream, an a„d,o 
so-eam, and possibly addhional conlen. stteams. 
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i^epacKager 2204 passes the kevs and rules to rep ^^ns 
receives DieiBox 27nrt in - i uics lo ituf 2205, and 

Di6.Box 2206 i„,„ u,e ,,,,, ob.ee, 0 " T " '"'"^ 

Repac.,. 2204 a.o a^J ™ '"'"^'^ 

DigiBoxe. a,e necessa^,. " "-'""tas -f -Olittonal 

Repackager 2204 outputs Proteced MPEG-l Ca„,e„, FiP. ,,n7 
Initial ObieclDescriMor 2705/ , ^- " ' 2207, cotuisling of 

Real Networks Implementation 

.nror.al::~^^^^^ ^ "^^^ - - ^ ~n w.th 

The R , M " '^^^""^'^^^ Networks I„c 

The Real Networks file fonnat (RMFF) is .llustrated .n FIG 13 TV / 
includes a block of header, .h. i, • ^^'^ 

content packet, (C™ r n''""'"' '"'^ '""""'^ 

(index , 03, Err *' ^ '""'^ -"^ ""d goto „pe„,i„„, 

*.e,.a.irpo:::z:.;T:rTr""^"'^^^ ^""-^ 

parameters for the decompressor) miomiat.on (e.g., 

r;:r;:rr:!t:;— 

includes enc^ted Content ,406 a^d Z 

Changtng the t>pe forces thTRea, N » '>"' '"-^ -ed. 

PIngtn is tegtslL ^T^T ' " ""■S'"." -"ce this 

proLted rii :::'r"t ^-^^ ^"-^^ ~ 

necded, aetet^tnesTe o ^^te^^^ ^'^ ^ - i' 

content, and then dec^sLl " , " ' """" 

--.o--~:::e::::::r"""^'''°'^'-^^^ 
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o—bo"!".™,. ,h. speoinc a„era,io„» „,ade ,„ a,e Real Networks file f„™„ 

are the lollowing: 

• U,c„.c *e pre.,, „ f„„ ^ ^ ^^^^^ 

an ■„c„a.e of 3 seconds is used. Larger buffers are needed beeause of d,e exrra steps 
needed to decrypt the content. 
. Modify eaeh s»ean,-specifie header by changing u,e nrime Ore .0 ••RNWK-Pn,.ec,ed" 
savrng .he old mime iype ,„ ,he decode, specific .nfomiadon and adding a comen, ' 
,den„fier and DigiBox ,o .he decoder speciOc infonna„o„. The DigiBox co„..ns .he 
key, ™„al,xaU„„ vector (IV,, ven,„n rnfomra.ion, and waiemtarking insirncions. The 
key, IV and co„,ent identifier are generated automatically, or can be provided as 
comn,a„d-,i„e parameters. The same key, ,V and content identtfier are used for every 
Stream. ^ 

. content packe,s are selectively encypied. ,n one embodiment, conten, packets whose 

5000 500) are encrypted. This encrypts approximately one-tenth of the contem 
reducmg encyptton and decryption costs, and damages U,e content, sufficentlv to 
prevent resale. The encryptton algonthm can be DES using output-feedback mode or 

any stmfar algonthm. The imtialization vector is computed for each packet bv xonng 
he stream s IV with the packefs star, time in m.llisecouds. Some tnformatiou uni,u! 
o the stream should also be xored into the IV. 1„ „„e embodiment, the same is used 

for mu^hple packets whenever two or more streams have packets wtth the same s,ar. 

..me. Thts usually happens for .he frs, packet ,„ each stream stnce they usuallv have 

star. t,me 0. Other th», the frs. packet, ,t is rare to have nvo packets have the same 

Start time. 

m one embodiment, these changes to the Real Networks file fomta. are 

Z7!!T,T ''""^ ^ " - in .he 

IT! ,r ™^ " "'^'^ Also passed 

to Packager 1502 IS Rights File 1503 Pack;,apr i ^n^i „ . 

^ ^^'^'^^ger 1503 generates Protected RMFF File 

1504, which .ncludes various alterations as descnbed above and as listed :n FIG 1 5 
mcluding the .ncorporat.on of one or more D.g.Boxes .n the header, encryption of the 
content, modification of the mime type, etc. 

in one embodiment, the trust plugin described above .s illustrated in FIGs. 16 and 
17. FIG. 16 .Uustrates the standard Real Networks architecture. File 1601 (e , a 
streammg audio file in Real Networks fomaat) is provided to Real Networks G^CIient 
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Ren.e„„, Device ,607^ Re„.e„„3 Z^ZZ T '° 
speakers, a ,e,ev,s,„„ ,ece,ver. e.c^ ^ "'""^ 

-Ne::::::~:r;r::'™^'^'"^"°^"-=-^ 

<.iurc. Mucft ot the architecture llustrateri in ftp it- .u 
that illustrated m FIG. 16 Thus File 170, ' ' ^ ^ '^e same as 

1702 through Setter 1703 or . n " ' "^'^"^"'^ ^"^^ 

'-.-.sthenpasse/to™^^^^^^^^^^^ 

IRP17I0. When uutially registered w,th Real Networks G2 Ghent Core 1702 T 
Plug.ns 1708 and 1709 infom. Real Networks G2 Ghent Core 1702 hauhe 
content of type RNWK-Protected WK n , ' ^''^^ ^^ey can process 

Protected. Whenever Real Networks G2 Client Core 1 702 

from ,he box for ,h , " """^ ^ey and IV 

SUBSTITUTE SHEET (RULE 26) 



wo 99/48296 

PCT/US99/05734 

-32- 

box Also, .his e„.u,e. ,ha. a use, ,s only asked ,„ pay „„ce ev=„ if ,Ke,e a,e ™,,iple 

s^s By sKa.„, e»e„. .,s. .eys, ^ IV, several ffles can ^ pUy^^u, 

, „ ^'^^'^^^"sed even ifanother stream with the 

content identifier has already been opened. 

If no other stream has been identified with tK» 
,708 r^- the same content Identifier, Trust Pluein 

1708 passes DigiBox 1404 to IRP 1710 IRPlTinm.vK a, "si nugm 

„ ^ ' ^ ^ ' ' 0 may be a software process running on 

.hesa™ec„n,p„,erasRealNe,„„,.sG2C.ie„.CoreandT™s,P,„gi„ 1708. IRP 17,0 
n^ay ™„ ,„ a p,„,ec,ed environ„,en, or ™ay ,neon,o™e ,a,„pe, resistance ,ech.,„es 
designedtorenderlRP 1710resislanttoanack. 

w .ch n,ay then be passed >o Tms, Pl„gi„ ,708. Tn.s. Plug,™ ,708 „,ay then use .his 
mfonna.io„ .o dec^i,. Enc.yp.ed Comenls ,406. 

Tms. P,ug.n ,708 uses .he o„g,„a, n,in,e .ype i„f„nna.,on cx«c.ed from Media 

con,e„, ,e.g, Rendenng Piugin , 70S, Once .his is done, Tnus. Plugin 1 708 behaves ,ike 

Ne™2 Chen. Core 1 702 passes seamed ,„f„nna,io„ ,o Trus. P,ug,n , 708 which 

deer,..s ,ha. nfo^ion and passes ,. ,„ Rendering P,„g,„ ,70. Pro. he pel pect „r 

Real .envorks C2 Chen, Core 1 702, Trus. Plug,n 1 708 co„s.,..es ,he appropnal 

rendering plum, and ihe core is no. aware fh,f,h ■ , ■ ■ 

Pluein,7n8,„. . . "'"'""""='"f''™»"<'"'»W"SPaasedbyTrus. 

Plugin , 708 .0 a second plugin (e.g.. Rendering Plugin 1 705) 

Similarly, from .hepo.n.of view of Rendering Plugin ,705. Trust Plug.n 1708 
enaves ke R=a, Networks 02 Cent Core ,702. T.„s ahhough Rendenng P, gL 705 
-ce,ves dccnT,.cd s.ream ,„fo™a.,o„ from Trust Plugtn , 708, Rendenng P „gm r7 

::roc:rd\:T:;~^ 

Network, CO rr n ^ ' '"''"'""^ ^''^^^"on Real 

Networks G2 Client Core 1702 or Rendering Plugin 1705 

Tnist Plugin 1708 may also perfonn other processing that may be helpfi., for 

secun^puiposes. example, T.st Plugm 1708 may watermark the deci^ted file pr.or 
to passing It to Renderine Plugin 1 70-; • . , ^ 

be such that It will . h '''' ^'g^^^hm must 

that It will survive decompression of the file by Rendering Plugm 1 705. 
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MP3 Embodimeot 

The techniques descnbed above can akn ■• . 

TT,^ vm , can also be apphed to MP3 streaming content 

■802. Cc„,e„, ,802 . .ivided ,n,„ L \ 

■nCude a ,arge „™,„ „f «- Con.en, ,802 „ay 

a„d ,3„r ■^a.H.de. ,806, ,807 

-™arrrrT:::;2r:;::Tr" 
Ka...^.«.,8,o;:,r: 8 t;:;r^^^^^^ 

• UnencryptedMP3Content 1912 This is the fir«:, f ' 
Playe. and wi„ be rendered by any standard ^^3 at^^ ^^^^^^^ ^"^""""^^ ^ 
user indicating that the confent • ^ ^ ^ ""^''^S^ «° t^^e 

bnes in each . J^Jo ^ ""^'^ ""'^ 

after ,h. header a„d CRC ,„7 « "J^- 

generated and then xored with ,h. fr " ''^"^"^'y 

Ma..a,.e™:r::::ra;:::~ 

- ^'reren. ene,^,,: Xr:" 

ID3 VI Trailer 1902, including 128 bytes. 
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1D3 VI Tra.to 1907. which is a copy of Trailer 1902 

Conten(I901.smce3,leaslaponio„oftotco„K„tisc„coT.led Sucha„l,v„ 
mos, likely read Ihrough to Trailer 1907 ™h i-ch a player would 

FIG. 20 ,l,us,,a,es one ertodrmem of an MP3 player designed .o process and 

"""=^'2007, and renders conient to Rendering Device 2008 In one 
en-^d^en, .s a nrodified vcrs,„„ of a player dis«h«ed hy So„i,„e 

Protected I^lT" T '^""^ '--^ 

u ivu-j Mie 2002 may have the format illustrated in FIG 19 

Trust Plu , 2 r' n ~ ™' ^005, 

n-st Plug^n 2003 calls Approval Functton 2009 ,o detemnne ,f Protected MP3 Ftle 2002 

s protected and whether authon^tton cists to play the file, App„.„ p „ 
Stven a potnter to Protected MP3 P„e 2002. I, then chec JLee Jk^ p To 

for the presence Of Trust ID 1906 If Trust m TQO/^ . ^^niezwi 
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^ . App^va, P„„c„o„ 2009 i„dlca.„, . „ ^ „„, ^ 

If DigiBox ,904 is opened i„ compliance „ift applicable ™les, ,he key and rv a„ 
retrieved and passed to Decrypt Function :>nn5 tu ,. . =yandivare 
, J f„, , , ^ riuiction 2005. The key and IV are stored with the content 

t^'"'' ^""^ ■= ™» -P- overa, 

sy^ etn P«fom,ance, s,„ce tt reduces the ntnnberof titnes a DigiBox m« be opened Each 

such action may introduce significant latency. =o|«ned. Each 

On the other hand, stonng this infonnatton in unp^tectcd memoty may teduce 
overall system security. Security may be mhm,^ .i, .. yreauce 
(,h„.^ • . ^ '^'^™""'='^"*="')">°l storing this infoimation 

and comenttd ate referenced when Approval Function 2009 fi,^ 
0 ec sC„„,c „,,,„3.odet=r.meifitma.chestheCo„ten.i;:fa„ar:^^^^^^^^ 

emtttal zed usmg the stored key ^ rv corresponding to the matching content id an, 
va e .n^, ^, ^ ^^^^ J ™ 

Approval Function 2009. 

Once Protected MP3 File 2002 has been opened, each time Player 200, needs a 
pac e,. Playe. 200, ,, ^^^^ ^^^^ ^^^^ J 

rema,mng data and a h^e number .o Decrypt Puncton 2005, wh,ch decrypts th tvl 
necessary, and returns it to Player 2001 . 

In a current en^bod.ment, although aud.o content :s encrypted, headers or trailers 
are not encrypted. This allows the Player 2001 tn nr. r ^''^rs or trailers 

^ *° process information in headers or trailpr^ 

without intervention from Approval Function 200Q nr n , c ■ 
Player 2001 to m. • . °' ^'"^^ 2005. This allows 

^yer 200, to place mfonnation such as playing time, artist and title into a playiist display 
and nutiali. Decompressor 2007. without any action required from Trust Plugin 200 

Commerce Appliance Embodiment 

This section «-nbe an embodimcn.. comprising a Commerce Appliance 

,^err ™"°' •'■^f-turesof^yo^ 

the embodtments provided elsewhere in this description. 

■" "ne embodiment, ,his section will describe modifications to the MPEG-4 
standard designed to suppott the avocation of persistent niles and controls wid, MPE0.4 
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In one embodiment, shown in FTH 9t ^ 
for govenung „e use „L " rn "^"^^ '"^ "^-1"= 

Each goven^ed digital work is associated with one or more CMPOs fContent 
Management Protection Object) eg CMPO^ c u ^ ^^^^ (Content 

(■■CCMPO-) used ,„ '• ''""''^"'•""■•"'"y specify a ChamKlCMPO 

assoc..:,. ,pji7:;r::::rrrr'"'°™™^°"'"-'° 

object goy^ng the pa„ic„,ar 

In one exe„,p,a,y app,ica,i„„, Comme„:e Appliance 2301 may be an MPEO 4 

player containing CMPS 2302 . may oe an MPEG-4 

satisfied CMPS 2302 Z ' . 

MPEG, wo* Zy ,hltTr H ™^ of 

storage location o ^^^^ '° "^^^"^ any 

B ^'"^^''''^''AVOs^ere actually released for viewing 

ao»n,oig~::r r --^ ^°™™- - 
co„,.nediieccro::::^t:;r^ 

MCMPOa.oc,a.ed....epa,..c.a;~^^^^^^^ 

urn supply a key for decryption of the associated ES 

ng distnbuted, peer management of content related rights by securely 
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applying rules and controls to govern the use of r^n, ♦ 
Appliance 2301 may make use of any of the mean, fi^rr, 

nnhiah. ^^^°'^P'^°^ectmg and using digital content 

on high capacity optical disk, in one non-limhing example a DVD \ 1 T 

aforemendoned Shear patent application. ' " ''^^ 

mana^emr? ^'''"^^ '''' specal-puipose fimctions relatmg to other 

n,ay also be de.,g„ed so .ha, ,3 ne^or^Me w„h oU,„ Co™„,„, .XTJce . 
a » . op .ox co^eced ,0 . OVD p,a,o. ^ , ^ J Ir^^^^^' 

onapen:::::"— ^^^^^^ 

— eco„p„.o,o...oj:::rd~^^^^^^^ 

ADDliance^ wh^r^ "^^^ ^ Commerce 

^ppiidncej Where the one or more CMP^<: f^fi^.^ n 

.he docking e„v„o™e„, ,„ ft™ a " " ^'^ 

iea^t in n,^ '^"^ '"'^'^ inter-operation through at 

heCo.™,e„eApp„.„cea„d,Ke,n.s,e„v,ro™„e„.,apa, es of ..edocKin, 

env,«, ,e.,, ^„,e. one o. .ore CMPSs and co„,e„, nsage .anage J, 

■nfonnanon, sueh as. for example, .nfonnation provided by use of CI) 

„„ , . , ffansraission, composilini and 

rendenngofv,deo and other types of informarion 

exan,p,r!fTr'r"" """^ ""^"""^ ^ -'-'."S 

ako use one or more CMPSs as described herein 

.aytncrrr":;:.:""'" 

type applications consisting of aggregated composite content 
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boxes, etc. '^'^'^"^ browsers, set-top 

Content Management and Protection System (CMPS) 

Each commerce apphance includes one or more CMPS (e g CMPS 2309^ tk 

Particular fimct,ons of CMPS 2302 include the followmg: 
(a) Identification and interpretation of rules 

wi,h . work. °' '='^0= 2303 a.socia,.d 

»d/oro*.,CI ,„ '';' '*™"^'"f''"""""'*<'"><>n-r more CMPOs 2303 

Objec, (-AVO-). ' """^ " "^'"'^ " Audio Visual 

(c) Decryption of conlem as allowed by the rules 

^ecr..rzrr^^^^^^^^ 

c«..re,.p,.eo^ 

.be content may be as desct^b^ by rEO-4 ' ""^ '"^ 

(d) Control of content based on rules 

Gintcr .333 patent appl.catto n le "^t^^^^ 

CMPS 2302 ev.r , of MPEG-4 systems, this may require that 

escnptors 2308), scene rendering (performed in Composite and Render 2309) 
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CondiUonal Access conttol ™,h 1 . ' <2) various 

deepen., in o,hcr publicaHo.- (3 l^ M ' 

proposed in tl,e Gi„i„ Management Language, such as Uiose 

,638 « o S efiH. =^ "'^'"^'^ 

Ma.e;,:;:c::'r,:^'~*'^^^^ 

serial ™ • ■ '^"8™''™. Lacy; (5) ,he CCI layer bits for IEEE 1394 

andT.«bap:;::~:e::r::r'^""*^^ 

(e) Monitonng use of content 

protected cont 1 Jd I T '""^ '° ^Vsteo, o. 

needed ,„t payntCCr "^^ ""^ 

(0 Updating user budgets 

— :t!::e::rs!:::r~ 
<«Ha.„arei..;rrc:::~""™^^^ 

process,oni!on°:::z::::::°:;"'^---------™e^^^ 

--di„.be.nco.po::x::.::;:*--^----"--^ 

0) "'"'"i-g keys, digital credentials, such as certificates and/or 

ctr ' '""r ''""'"^ 

cleannghouses, and/or other trusted infrastructure services 

a«nMeinl:r'"'""="'*'"'«=-^----'^"-P«-»P-»«H"S-or 

«J:ix^:^::::;-"------»-."ests 
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(m) Securely certifying or otherwise guarantedng the authenticity of 
apphcat.on code, for exa„,pie certifying within CMPO 2301 and/or CMPS 2302 that 
application code containing rules and/or other application information, such as infoimation 

wnttenm Java code for conditional execution within a ron,n, a 

"^"""^ a Commerce Apphance, and/or that 

a, leas. ,„ pan „„,3i<,e of CMPO 230, and/or CMPS 2302, has „„, b«„ a,.er«, 
and/or has been delivered by a gua^teed (e.g., ™ed) party. 

(n) Securely processng independently delivered CI, such as descdbed in U,e 
.nc„ Ointer .333 paien, app,ica,i„, ,0 perform content usage cent,., that protects 
the nghts of plural, tndependent parties in a commerce value chain. 

(o) S=-'=lyP<rf>n™ngwatem,aridng(incl„ding,forexamplcfmge,p,uiting) 
Itacnons, for example as descdbed in the Ginter '333 paten, application and as 
.ncorp^rated hcretn, for example including i„ten,re,ing watennarKing tnfonnation .0 
control content usage and/or ,0 ,ssue an event message, therein such event message may 

be reponed back to a remote authotity, such as, for example, a MCMPOHghts 

cleannghouse management location. 

of ,H r*^' '° "^"""^ """" '™ configuration 

of . e Conmter. App,i.„oe and any connected devices <e.g., which loudspeakers L 

apa^le.,de„,.fication„fattachcdmomto,s, including whetherpat^cularmonitot^h 
g..l output pom, etc.) ,f anached devices (such as loudspeakets, also include CMPSs, 
CMPSs may be used ,0 commu„.c„e for purposes of cootdtnation (e.g.. a CMPS m a 
set-top box and/or loudspeaker an^gement may c„mmu„,c«e w„h a CMPS in a 
^iownstream d,g„al television or other display device .0 establish which CMPS will be 
n^ons,ble for goventance or the nat^ of cooperative govemance through a virtual nghts 
process, satd process opttonally i„v„,v,„g a rights authodty server that may find, locate 
provide, aggtegate, distrtbute. and^or manage rights processes, such as described in the ' 
aforementioned Shear paten, applicatton. for employing pluml CMPSs, for example for a 
sragle u!«, content processing and usage atrangement). 

The present mventton includes anangementscompnsing plural Commerce 
Appliances ^or CMPSs in one or more user locations, non-limiting examples of which 
.nclu e home, apanment, lo,,, office, and/or vehtcle, such as a car. tntck, spom ufility 
vehicle, boat, ship, or aitplane, that may communicate among themselves at least 
occasionally and may comprise a vimtal network that operates in a logically coopenttivc 
manner, through at leas, in part the use of such CMPSs, to ensure opfimal commercial 
flex,b,hty and efficiency and the enforcement of nghts of conmetce value cham 
panicpants, includmg financtal and copyright rights of providers, inftastntcture rights of 
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to such mfomation thioueh ai 1,=., ^ '"""S "S^t 

toc,.o„.„, of VDE aod" °° "^'"^ '^'""■"S 

In one embodiment, Shown in FIG 24 CMP9 9401 

software. '''^^ '"'^'"dmg execution of any necessary 

(b) One or more exten^ai commumcations ports, e.g Port 2403 Pon .sn. 
commumcates with External Network 2404 wh.Vh • . . 

cMPs B,os ..s, <a, ...,co j::,~i^r:T™ '^^^ 

(3) Control Pnmidvcs 2410 whi.h ^ P=™a„en,ly stored in CMPS; 

a-oc.a,.d „„h ,h cl^sl ?H ' " """""'^^ <^> '''^^ ' 

Certtaes 24, H !' ' " '■"*'Wva,e Key Pair; ,5, „„e or mo,. 

»-enuicates 2412 designed to dentifVCMP<; 74ni .u . . 

.-o™a.ioM.>Har.„are..aJ,ZZ~^^ 
^he<,.,a„„r.„.,„,,,,.,,„,,^_^^^,i--^^^^ 

Objec. Id.n,if,ca,j„„ 2417 of ,h„ cun^mly acl,ve; (3) Co„,e„, 

24. wMcb are cl: : :' ^^<^>^ 

-™.-o..eoj.ea„j:::::;:~^^^^ 
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state of composition and rendering); (6) Stored Exhaust Infonnation 2420 relating to use 
and/or the user, designed for external transmission; (7) Updated Budget Infonnation 2421- 
(8) Content 2422; (9) Active Content Class Information 2423; and (10) Active User 
Identification 2424, including identification characteristic infonnation. 

i»- NVRAM 2425 (e.g., flash memory). This type of memory may 
hold information which is persistent but changeable, including at least some: (1) Budget 
Infonnation 2426; (2) User Infonnation 2427, such as identification, credit card numbers- 
prefen-ed clearinghouses and other Commei,:e UtiUty Systems; (3) User Preferences 2428 
such as preferences, profiles, and/or attribute infonnation; and (4) Appliance hifonnation ' 
2429, such as attribution and/or state infonnation. 

The types of information descnbed above and stored in CMPS Memory 2405 may 
be stored in alterative of the above memory types, for example, certain budget infonnation 
may be located in ROM, infonnation regarding specific one or more clearinghouses may be 
stored in ROM, certain active infonnation may be moved into NVRAM, etc. 

Budget infonnation may include stored budgets made up of, for example: 

(1) electronic cash; 

(2) pre-authonzed uses (e.g.. based on a prepayment, the user has the right 
to watch 12 hours of programming). 

(3) Security budgets related to patterns reflecting abnonnal and/or 
unauthorized usage, for example, as described in the incorporated Shear 
patent, wherein such budgets restrict and/or report certain cumulative 
usage conduct. 

(4) eiectromc credit, including credit resulting from usage events such as 
attention to promotional material and/or the playing of multiple works 
from one or more classes of works (e.g., certain publisher's works) 
triggering a credit or cash refimd event and/or a discount on fiiture 
playing of one or more of such publisher's works, such as other works 
provided by such publisher. 

User infonnation may include the following types of infonnation for one or more 
authonzed users of the Commerce Appliance: 

( 1 ) Name, address, telephone number, social security number or other 

identifier 

(2) Infonnation used to authenticate the user, which may include a user 
selected password and/or biometric data, such as fingerprints, retinal data, etc. 

(3) User public/private key pair 
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(4) User attribute and/or profiling infonnation. 

iv. Removable Memory 2430. This may include any type of 
removable memory storage device, such as smart cards 
floppy disks or DVD disks. If the commerce appliance'is 
designed to play content received on removable memory 
devices (e.g., a DVD player), that capability may be used for 
purposes of the CMPS 
Memory 2405 .ay include a p,<„ec.ed database, ,„ whick certain conTo, budget 
secunty. and/or cryp,„grapbic inr„n.,a.i„„ is stored in .ecnre memoty ITjl^ 
mfonnatton stored in an encrypted fashion in ^secure memory "P'* 

recemng CMPS may be employed to control such content's usaue in.l„H- f 
tetypting such content, as appropriate Encrvn, /n 

a Random Number Gen rator ^33 u . 2« ' -ay include 

used to tdentifv and t ' '"^^ " "e 

-ed tdenttfi, a„d assure the uniqueness of CMPSs and suppor, the opening of secure 

—.onchamrelsbetweensucbsecurecontentcontrolsecuree: 

n wr, Z""''"'''''"''"''"^"''" 2434, CMPS 2401 may include Secure 

C e^Ca endar 2434 designed to provide absolute informatton regarding the dal and time 

m2^^:^Z Up 2435. 1, may ^her include Syuc 

Mechat^sm 2436 for synchronization with outside timing .nformatton. used to .covC the 
cotrec, ,m,e m the event of a power loss, ^d/or to cbeC for t^peHng 

(0 Interface 2437 to blocks used for content rcnderiog and display This 
m^ace ,s used for controlling rendenng and display, based on nrle. and fo olim^g 
^ac mfo^ation, which may be used for budgeting purposes or for provil" g ' 
information to outside servprc ^ r p^^vming 

wh^chchoicestheji: :::r:;r^^^^^^^^^ 

mvoked, etc.) In the case of an MPEG-4 player such as is shown in 
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example, buffenng, u,e ^ <|=«^p„, g^,, ^ vo decode. „b,«. descriptor, and 
compose apd rendering (e.g..Co„ttolL^es 2310. 231 land 2312) 

2302 , r'"' CMPS 

c"™"° '"^^ -'-^ vrewer. For 

example Compos,,, and Render block 2309 can issue a s,a« even. ,o CMPS 2302 when an 
AVO o,ec. ,s released for v,e.ng, .d can ,ssne a s,op even. .„ CMPS 2302 wKen Z 
AVO object IS no longer being viewed. 

Feedback Iron, Composuc and Render block 2309 may also be used .„ de.ec. 
■anrpenng, by altowing CMPS 2302 .„ ma^h U,e idenSficadon of Ure object acmally 
..leased fo",ew,ng wiu, U,e idenMcaUon of .he objects auUK,rized for release sjand 

indicative of the occurrence of an unauthonzed event 

stare <-d>T "^^^ '^^^'^^^ ^-a: 

start <.d>, T, instance numberxclock timeXrendering options> 

Sent if elementary stream <id> is reachahlp in th« cr. 

■ , reacnable in the SD-graph at time T, but not at 

time T-L 

»d <ld>, T, <l.su,nce numberxclock timexrendering „pH„.s> 

T co„s„„„es presemaUon „™e, clock .tae consd.„,.s *e wall Cock .in,e, ,„cludi„g day 
and d«e ,nf„™,o, and .„den„g „p.i„„, ^„,,„,^ ^^^^^^^^ ^ ^ 

Of play (e.g., fast forward). 

..mc r A sr'T"^ '"^^^ « "« - 

nda,!' 1 T °f ^^-^'P" 'or display 

CriT^ ""^ ' ""'^ ^^^^^ <'-^> 0 

modtficd Tl„s .mphes al, nodes in ^ „^ „eed ^ „pda.e his^ry lis.. This lis. need 
no be 3S large as .he number of steams. Purther, ,. can be labeled .o^„d,ca.e if .he C^S 
-.1 be wa.ch.ug for s.ream. if no, labeled ,. will „o, r^ord d,em. An AV eleme^Iy 
.mam ,s reachable if ,he seam's comeu, was rendered. 

For SD-graph update sttean,s. .he objec, insiance number is ignored. For A V 
^.can,s. the mstancc „™,ber can be used .o d,s=mb,^a.e the case where the display shows 
»o or more mstances of the same data st^am s,mu,.aneous,y. ^stance nmnbe,. do no 

stt:::;, r " r ^ '--^ - '^'^^ - » 

sian event with an end event. 

ha second embodiment, CMPS 2302 may include some special purpose hardware 
m combmanon ™,h general p^se h^„.„ which is also used for od,eI.ctio„s ofTe 
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device. In this embodiment, care must be tat tn « 
Wions are performed . a s..^Z 2 '''' 

purpose hardware Each of th eT . '^^^^^ - of genera. 

fiZn . i^'^l^de dedicated CMPS 

fiinctions and general purpose device fimctions: 

(a) CPU/microcontroller. This mav mri.,^« 
thflnnn^^o • , ""^^^ynclude one or more devices. If more 

rend^ed «„per.,«,s,a„, or U,. devices „.y co™„^,e <,„ a .ecure bus T„c CPU 
mclude two modes: a secure CMP*; j me CPU may 

secure CMPS mod T '^''"'^ '"^'l-- The 

r "''"""'^ --OO' locations unavailable to the 

=::r:z:re.-::;r^^^^^^^ 

. o J '^"luty space, so that, m unsecure mode the CPJ t 

caraiot address secure memory locations 

operate a Co Jl^rT """^ " - 

tor CMPS operation that would emolovNfVP A iv/f u , yoeneeaea 

be accomplished i„ of .he K ™^ 

o„ "^^^y^'Ormacombmationofthesewavs- (U 

^ra;:r;,r'"""'""-"°"''"''''°«-'^'--- 

f u- ^ ^^'"^°""^^onmaybestoredinanencrvnteH 

fashion, though this requires at least some RAM to he 

direct acce« . . ''^^ ^MPS will require 

direct access to unencrypted information stored in RAM. 

inCud- . E"-^^>on/dec.^t.on engme. Encryption and decryption ftmctions 

n.ud.ng icey generation, may be handled by special purpose softwarel.„g on ge^^^, 
purpose processor arrangement, particularly for examole a fin . 
DSP arTanapm<.„t Tu . ^'^^ '""^ ^''^P'^' ^ floating point processor or 

DSP arrangement. That processor arrangement may also be used for purposes of 

-ompressmganddis,ayingcontentan.or for handing watenna^^^^^^^^ 
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« and/or reading. A,,en,a.iv.,y, ,he devce may mcl-de nalive e„c,yp„„„ and 
ypuo. tacaons. Fo, example, vanous emerging ..andards may „,„lre a, leas, some 
de^ee of enc^ion and decrypiion of co„,e„, designed .o be passed ac^ss u„.ecu,e buses 
™inn and among deviees such as DVD piayers, sueh as ihe "dve company proposal" and 
odier lEEE ,394 re,a,ed imdauves. Cireu,»y designed ,o perform such encrvp,.o„ a™, 
decrypnon may also be usable for CMPS applicalions. 

<e) Sccme cloctealendar. The underlying device may already require a, 
eas, some clock informal. MPEG^. f„, example, requires d,e use of clock informadon 
for synchromzation of ElemenUry streams. A secure CMPS Cock can also be used for 
such purposes. 

In a ihird embodiment CMPS 2302 can be prinrarily software designed ,o nm on a 
general purpose device which may .nclude cerram mimmal sec„ri,y.rela,ed feamres In 
such a case. CMPS 2302 may be rece,v^ m ,he same channel as O-e conren,. or ,„ a side- 
band cham,e,. An I-CMPO and/or od,er CI may specily a pardcular ^e of CMPS. which 
Commerce Apphance 230, mus, e,.ber have or ac,u,re (e.g., download ftom a loca„on 
speeded by U,e I-CMPO), or CMPS 2302 may be included, for example, „,U, an ,-CMPO 

A software CMPS nms on CPU of the Commerce Appliance. This approach ' 
may be ,„he,en,ly less secure U,an d,e use of dedicated hardware. If ihe Commerce 
Appliance includes secure hardware, ,he software CMPS may consulate a downloadable 
OS and/or BIOS which cusiomizes ,he hardware for a particular type of commerce 
application. 

In one embodiment, a software CMPS may make use of one or more software 
tamper resistance means that can matenal.y "harden" software. These means include 
software obfi.scat.on techniques that use algonthmic means to make n ..ry difficult to 
reverse engmeer some or al, of a CMPS, and further make it difficult to generate from a 
reverse engineering of a given one or more CMPS. Such obfi.scat.on .s preferably 
mdependent of source code and object code can be different for different CMPSs and 
d.fferent platfonns, adding fi.rther complexity and separation of roles. Such obfiiscation 

"^'^ - an CMPO, as well as to some or al. 

f the CMPS .tself, thus obscuring both the process.ng environment and executable code 
for a process. TT.e approach .s also apphcable for mtegrated software and hardware 
™plementat.on CMPS implementations described above. Other tamper resistance means 
can also be employed, including using "h.dmg places" for storing certain state mformation 
in obscure and unexpected locat.ons, such as locations in NV memory used for other 
purposes, and data hiding techniques such as watermarking/fingerprinting. 
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Association of CMPS With a Commerce AppUance 

A CMPS may be pennanently attached to a particular device, or may be partially or 
fiiUy removable. A removable CMPS may mclude software which ,s securely loaded mto a 
Commerce Appliance, and/or removable hardware. A removable CMPS may be 
personahzed to one or more particular users, mcluding user keys, budget informatiort, 
preferences, etc., thereby allowmg different users to use the same Commerce Appliance 
without commmgling budgets and/or other rights, etc. 

A CMPS may be designed for operation with certain types of content and/or for 
operation with certain types of business models. A Commerce Appliance may include 
more than one type of CMPS. For example, a Commerce Appliance designed to accept 
and di^lay content pursuant to different standards may mclude one CMPS for each type of 
fonnat. In addition, a Commerce Appliance may include a CMPS provided by a particular 
provider, designed to preferentially display certam types of content and to preferentially 
bill for such contem through a particular channel (e.g., billing to one or more particular 
credit cards and/or using a particular one or more clearinghouses). 
Source of Rules 

The CMPS must recognize those mles which are to be applied to particular content 
Such mles may be received by the CMPS from a variety of sources, depending on the 
particular embodiment used: 

(a) CMPO. The mles may be included within a CMPO (eg CMPO 2303) 
and/or other CI. The CMPO and/or other CI may be mcon^orated withm a content object 
or stream (as, e.g.. a header on an MPEG-4 ES), and/or may be contained within a 
dedicated content object or stream encoded and received as per the underlying standard 
(e.g., an MPEG-4 CMPO ES), and/or may be received outside the nonnal content stream 
m which event it may not be encoded as per the underlying standard (e.g., a CMPS 
received as an enciypted object through a sideband chamiel). 

(b) CMPS. Rules may be permanently and/or persistently stored withm a 
CMPS, e.g.. Rules 2409. A CMPS may include default rtiles designed to handle certain 
siniations, for example, where no CMPO and/or other necessary CI is received (e g 
content encoded under an earlier version of the standard which did not incorporate CMPOs 
mcluding MPEG-4 version 1). Complete mles which are stored withm the CMPS may be ' 
directly or indirectly invoked by a CMPO and/or other CI. This may occur through the CI 
Identifying particular mles through a pointer, and/or it may occur through the CI 
.demifying itself and the general class of control it requires, with the CMPS then applying 
particular mles specific to that CMPS. 
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Rule "primitives" may also be stored within Uie riUP9 n 
7410^ Th^rA/roi^ . ™" (e.g.. Control Pnm lives 

-4.0,. T^e CMPO 3„d/or «her CI .ay ,„v„k. tee p.n,Wv. ,y , ,.,„^,, 

llowe^ Examples ,„c,„d.: (a) ™le. desired .or=,„i,e,ha.c=„ai„,yp.,„f con.™ 

»c,«.^ body such as a govc^enc agency,; ,b) ™,es designed ,„ re,„i„ d,a> o„,y 
^a^.„,a. .e aUowed ,o invoke open„io. ,e,^g pay„,e„, ,eyo„d a cc J„ ,i„n, 
and/or aggregate payment over a cenain amount. nam l,m« 

The user may be allowed to create templates of ndes such as descnbed m U,c 
afo^menttoncd Ointer .33 paten, application (and tncotporated heretn,. 1„ addit^ a 
C^^S=^g^e„t.and,orap„icu,arCMPOa„d/o,od,era,mayrestr,c,tha,u,. 

(o to h able ,„ v,ew. hu, only after a parent to the Hrst user, User supplied on. or 
more ™les may govern the use of - mcludtng pnvacy ^trict.ons relatedl- paymen 

proHhng. preference, and,or any other kind of info^atton (e.g.. infonna oTI ,. ^ 

contirr:; 

ntentl^Such user supphed one or more rules can b. assoctated w„h U,e user and/or one 
or more Commerce Apphanccs m a user arrangement, whether or not the infonnation is 
~ according to one or more cntcH. and whether or not use, and/or applil 
d»..cat,on ,m-onnat.on ,s removed dunng aggregation and/or subsequent re^I„,n 
dismbutton.oranyotherkindofuse. '^"""g, 
Tl,e ability to allow the user to specify rules allows the CMPS to subsume (and 
e.by rep ace, V-ch,ps. smce a parent can use content rattng infonnafon to spec" 
precsely what types of info,nnat,on each v.ewer will be allowed to watch (e g viol 
cotttent can only be displayed after entry of a certain password and/or other .Lfier 
ncMtng. for example, tnsernon of a removable hardware ca„l (sm^ or Hghts ca^^^' 
possessed by a user). ^ 

^""-^ The mles may be stored on an extemal server 
Rui smaybeaddressed^ddownloadedby theCNO-S if ne^ssary (e g eitherthe cZ 
and/or Other CI and/or the CMPS rnnt.in , eitner the CMPO 

CMPS contains a pomter to certam rules location(s), such as one 
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or more URLs). In addition, content providers and/or clearinghouses may broadcast rules 
designed for general appUcability. For example, a content provider might broadcast a se, 
of rules providing a discount to any user participating m a promotional event (e.g.. by 
prov,dmg certain user mformafon). Such rules could be received by all comiected devices 
could be received by certam devices identified as of interest by the content provider (e g 
all recent viewers of a particular program, as identified by exhaust information provided by 
the CMPS to a cleannghouse and/or all members having certain identity characteristics 
such as being members of one or more classes) and/or could be posted in central locations. 
Example Embodiment 

In one embodiment, a set of MPEG-4 Elementary Streams may make up a work 
TTie Elementary Streams may be encrypted and multiplexed together to form an Aggregate 
Stream. One or more CMPOs may be present in such stream, or may otherwise be 
associated with the stream. Options are as follows: 

1 . Content may be streamed or may be received as static data structures. 

2. A Work may be made up of a single stream or data structure, or of many 
separately addressable streams or data structures, each of which may constitute an Object. 

3. If ^ Work is made up of separately addressable streams or data structures those 
streams or data structures may be muhiplexed together into an Aggregate Stream, or may 

be received separately. 

4. If streams or data structures are multiplexed together into an Aggregate Stream 
the streams or data structures may be encrypted pnor to such multiplexmg. The Aggregate 
Stream itself may be encrypted, whether or not the underiying streams or data structures are 
encrypted. The following possibilities therefore exist: (a) individual streams/data 
structures are unencrypted (in the clear), the Aggregate Stream is unencrypted- (b) 
.ndividual streams/data structures are unencrypted pnor to multiplexing, the Aggregate 
Stream ,s encrypted following multiplexing; (c) individual streams/data structures are 
encrypted prior to multiplexing, the Aggregate Stream is not encrypted following 
multiplexing; or (d) individual streams/data structures are encrypted prior to multiplexing 
the Aggregate Stream is encrypted following multiplexing. 

5. A CMPO may be associated with a channel (CCMPO), a work (MCMPO) or an 
individual Object (CMPO). 

6. A CMPO may be received prior to the controlled data, may be received 
contemporaneously with the data, or may be received after the data (in which event use of 
the data must wait until the CMPO has been received). 

7. A CMPO may be received as part of an Aggregate Stream or separately. 
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to. .H Stream, h n^^^ 

together with the individual streams or data stn.rh.r^ • 

or data structure. °^ '^""^'^^^ ^ ^^"^'^ 

9. If a CMPO is multiplexed within the Aggregate Stream, it may be encrypted or 
nonencr^ted. If encrypted, it may be enc^ted pnor to multiplexing, and/or enc^ted 
after mult,plexmg. if the entire Aggregate Stream is encrypted 

10. If a CMPO is received as pan of the Aggregate Stream, it may be (a) a part of 
he s^eam or data structure wl^ch holds the content(e.g..aheader);(h)aseparates.^^ 

H mT '° - ^^'^-^ - ^ structures 

which hold the content (e.g., an MPEG-4 ES) or (c) a separate stream or data structure 
encoded under a different format designed for CMPOs. 

11 If a CMPO is a part of the stream or data structure which holds the content it 
-y be (a) a header which is received once and then persistently maintained for control of 
heco„.ent;^aheader Which isreceived at regul^mte.^^^ 
structure; or (c) data distnbuted throughout the stream or data structure 

decrvnt'^'r rr """" demultiplexing and 

decryption of the CMPOs. FIG. 25 illustrates the following embodiment- 

Aco J'l^^^f ^'^^-^^^^ -«^-Pof-ltipiexed ESS (e.g.,ES 2502 and 2503). 
Acombinationofsuch ESS ma.es upasmglework. Aggregate Stream 2501 .generated 

byacableag^g^or and received byause.s set-top box asoneofanumberofc^^^^^^ 

.50S """'^""''"^ '° ^^'^^ '^^--^l --ent along the cable m Header 

-505 a regu ar intervals (e.g.. once per second). When the set-top box is nimed on, it polls 
each chamiel, and downloads all cu.ent CCMPOs. These are stored persistently, and are 
changed only ,f a new CCMPO is received which differs from prior CCMPOs 

CCMPO ' "^■'^'^ ''^'^ ^he associated 

CCMPO. The CCMPO may specif, for example, that content m this particular channel 
may only be accessed by subscribers to the channel. A CMPS withm the set-top box 
accesses a user profile persistently stored m NVRAM and detennmes that the user is a 
subscnber. The CMPS deems the CCMPO rule to have been satisfied. 

4. The CMPS obtains an identifier for the MCMPO associated with the work 
(video) cuiTcntly streaming on the chamiel and a key for the MCMPO. If works are 
received senally on the channel (e.g., a television channel in which one work is provided at 

any MCMPO currently on the channel. 
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5. The CMPS begins demuxmg of Aggregate Stream 2501 (this may occur in 
parallel w... the preceding step), and obtaans the MCMPO. which is encoded into an ES 
»ed w.th,„ the Ag^egate Stream (e.g., MCMPO 2506, ^though each ES w,thin 
A gregate Stream 2501 has been encrypted. Aggregate Stream 250, was not encrvpted 
foHow,ng mu.t.plexmg. Tlus allows the CMPS to demultiplex Aggregate Stream 2501 
without decrypting the entire Aggregate Stream. 

6. The CMPS identifies the ES which constitutes the MCMPO (e g ES 2503) 
The CMPS downloads one complete instance of MCMPO 2506 mto an intema, buffer' and 
uses the key received from CCMPO 2504 to decrypt MCMPO 2506. 

7. The CMPS determmes which rules are applied by MCMPO 2506 MCMPO 

2506 might, for example, mcludean,lestatingthat the user can view the associated work 
^^th advertisements at a low fee, but must pay a higher fee for viewmg the work without 
advertisements. 

8. The CMPS generates an options menu, and displays that menu on the screen for 
the user. The menu specifies the options, including the cost for each option. Additional 
options may be specified, including payment types. 

9. The user uses a remote control pointing device to choose to view the work at a 
lower cost but with advertisements. The user specifies that payment can be made from an 
electromc cash budget stored in the CMPS. 

,n Nwl'"?'^' """" """S" P---«y ^'O'^i 

".NVRAM. a«i generate. ^ enc a message ,o a server a.s«;ia,ed wUh ,he eable 

The message ,n„,sfen ,.e required budge, ,„ ,he server, e„her by ,,^fe,„„g e,ec„o„,e 
cash, or by au,honzi„g a financial clea^nghouse ,o transfer ,he an,„un, fton, ,h. user's 
accoun, ,o ,be cab.e p,„v,de.s. TWs „,essage ™y be sen. i™„,edia.e,y, or „,.y ,e 
Mere ,o be sen. ,«er (e.g.. wben ,he user eo„„ee.s ,b. dev,ce ,o .he teeme,,. Tins s.ep 
may be ,aken ,n parallel with decryption of ,he coment.) 

1 1. nte CMPS obtains from MCMPO 2506 a set of keys used to decryp, the 
Elententary Streams assoc,a«d with the work ,e.g.. ES 2502). Tlte CMPS also obtatns 

TT Tr;?'"' ''^ "^^ '"^^ ^"-'^^ «■»• nts 

^ .0 be tncluded, the MCMPO identities ESs assoctated with the advertisements, and 

.denttftes a Scene Descriptor Graph which tncludes adve„,semen,s. A Scene Desenptor 

G«h does not include advenisements ,s not identic. „d is not passed through by 

12. The CMPS passes the dectypted ESs .o .he MPEG-4 butfers. The nonnal 
process of MPEG.4 decoding, con,p„s,.i„g and re^iehng ,he„ takes place. The Composite 
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^^R»^, ,„ck „„,„, S,„ an, S.OP even,. f„, each objec. .^^^^^ 

CMP conta. „e adven.se™. have been „,=a.ed for v.ewing. and ha, eacl' 
opera»on ha, occup,edapp™e,yU,eexpec,ed =,„<,„„, of „„e 

In ancher en,bodi„,e„,, a se,.,op box con,ain,„s a CMPS (e.g., CMPS 2302 fro™ 
no have a cable upu. (e.g.. ean^ng M4 B« S„ea.s 23 ,4 and CMPOs 2303^ 

ehanne, ean^ng MPEG^ ES. ,e.g.. M4 Bi. S.ea„,s 2314), and *e o,her 
carrymg CMPOs (e.2 CMPOs tu u ^ sud cnannel 

ic.g., ^myus 2303). The sub-channel carryins CMPOs ?im ^«,.m k 
routed directly to CMPS 2302 with the , u ''^ 

o zjuz, with the ES channel being routed to a decryption block 
o^..ng .der eo„.o, of ,he CMPS. e.g., C^u 23,5). and ,hen ,o .heTeO^t ffe„ 

Dip,";r:r'"T"^"^'"°-^^-'-°°---,.dC 

Cht r ""'hanged 

ta,»lh d«,^„o„ b,„ek and ,n,o ,be buffer. Th,. .ay oecn,, for exan,p,e, ,f ,he ESs 
1^ be.„g hroadcas, f„, free, .W* no ,esn,c,i„„s. and/or if , hey are pnbhc dol „ 

CMPO r """"^ =y"Ch™niza,io„ i„fonna,ion in U,e 

CMPO sub-channe , so ,hal CMPn, i„ i. 

""-^^O^ 'an be syncI,ronizedwi,h,he associated ESs 
The concep, of i„con=o«,ing nvo separa,e sttea„,s. one cons,s„„g of confrol 
.nfon„a,.on and conneced d.rec„y ,o ,he CMPS. and ,bc o,her cons,s,i„g of ESs ly 

rptZbZirrc^^^^^^ 

^S^*''^C^Ofo™at without the necessity for 
.efo™a„,„g con,en, ESs. To ,ake a„o,her example, i, may be p„ss,b,e ,„ upgrade a 
Coerce Appi.ance by inc,ud,„g a „„v or differ CMPS. wi,hou. ,he nec^si. for any 
h^g. .0 any 0 , he circunry des,gned ,o den,n„ip,ex. co.pos„e and render ,he ;o„,en7 
ESs, A user n,,gb, obuin a CMPS on a sn,a„ card o, ote removable dev,ce and p,„! a, 
device into a Commerce Appliance Thi= .™ u k a ^c, ana plug that 

PP"^"- T^s<=o"'d be done to customize a Commerce 
Appliance for a particular application or for particular content. 

CMPS Interface to a CE Device 

'^ '^M'S may be designed ,opresen, a s,anda„iized,merfacebenvecn, he general 

P^o»^cno„a,i.y„facons™ere,ec^n,cs device .danyreievan, CMPOS a^r 
o*er a and p™,ec,ed con,en,. For example, a CMPS could be designed ,o accep, CI L 
™ HSs. and ou,u, dec^,ed ESs i„,o ,he device, bu.ers. In su!::::: ,he 
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n,=„„facn,^„f „„„„ ^ ,^ 

r : : '^'^^ AU such ex,a„s,o. 

would be handled by the CMPS. 

Initialization 

Initiahzatin n of the r\fP<; 

'^'^'^^ .he capabililies of the Commerce 

Apphance m wh,ch a CMPS is installed. A CMPS pen„a„e„,ly assoc.aled with a 
particular Commerce Appliance may have such infonnation des,gned-i„ when the CMPS is 
.m«al.y tnstalled (e.g., stored in ROM 2406 shown in P,G.24,. A CMPS which is 
removahle may he used to nu, an inttialization operaUon ,„ order to ohttun infonnation 

NVRAM 2425. Altemanvely. some or .11 of such information may be gathe^d each time 
the device ;s turned on, and stored in RAM 2414. 

For example, a DVD player may or may not oontatn a connection to an ext=n,al 
^rver and^r process. A CMPO and/or other a stored on a DVD (»d/or any other forma. 
o™„^rted into a DVD ,or any other format optical dis.) player may include 

f user dcnttftcatton tnformahon .s output), or may re^utre a direct co^tection ,„ order, for 
example, to download Keys used to dectypt content, h, such a case, the CMPS arrangement 
may dcermme the hardw„e mnct.onality whtch is expected by orre<,u,red hy the cLo 

^or other CI re<,u,res a network comtecfon , and that the DVD plaver does not include 

ch aconnectton. the CMPS may take a vanety of steps, tncluding: („ if the network 
connectton ,s te<,mrcd for some options but no, others, causing otdy those opttons which 
^e posstble to be displayed to me user; (2) .nformmg the user that necessary hardware is 

reason for the rejection. 

Which 1 "'.r""" ' ™^ ' "-"O" 

whtch allows the user to choose among quality levels (or other forms of vacations of a 
g™ work for example, longer length at,d/or ^ater opttons), with a lugher price bemg 
char d ,f he user selects a htgher level of ^ualtty (e.g., mustc may be played at low 

si a " Z r ' °* '° ^ ^''^'^ " » '-'"-°n). >n 

lu t ''"^ A""-" "■">■ « loudspeakers which are capable of 

utput mg sound at the higher rcsolutton. ^e CMPS arrangement preferably ideles 

.K.S sttuanon, and e.ther el.minates the htgher resolution output as an optton orthe user, or 
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info™,s the u«r ta. d,is op,io„ »s„ „,„,e bu. provides no addi.io,., benefu g,ve„ ,he 
Co^erce Appliance. co.e„, tac„„„ali,y o, given .Ke C„n,.e,ce Appiiance „„, be.ng 
doclced ,n a user arrangemen, ,ha, provides higl,er quality loudspeakers 

If the Corameree Appliance may be hooked up ,o extenral devices (e , 
loudspeakers. d,splay, eic). ihc CMPS w,U require so.e .echan,sn, for ,den,i W„g a«, 
reg,s<ermg such devices. Each device .ay be used ,o ™ake standard ,D and capabUitv 
.ntom,at,on ava.lable at all ,i™s, thereby allowing tbe CMPS to pol, al, connected devices 
regular tntet^als. including, for e.^ple, authenticating CMPS an^gentents .ithin one 
or m re of each such con^ec.ed devices. Ustng auotber approach, aU devices could be used 
0 output CMPS identification ,nfo„„atio„ upon po„er-on, with later co^tected devices 
being used to ouiput such infonnation upon establishment of the connection Such 
■dentiiication infotntatton .ay ,ake ^ fotm, fcrexantple, of authentication infonnation 
P^vided under the .five company a^ngemenf, such authent,cat.on .cthods are herem 
incorporated by reference. 

AS discussed earlier, a Commerce Appliance may be connected to multiple devices 

eachcontaimngitsownCMPSamngementtes a nvn „i, u 
di„it»lTvw 1. ''^"""'t«-8-''D™Playermaybecomiectedtoa 
<i.8..al TV, in such cases, the CMPSs mu.t be able to initiate secure conununtcation ,e g 
using a scheme, for example, like the "five company proposal- for IEEE 1394 serial bus, ' 
^ dete^ne how the CMPSs will ,„t.rac, wuh respect to content communtcaUon 
between CMPSs and, in certain embod.ments, .garding cooperative governance of such 
content such as descnbing i„ tbe mcorporaied She. patent application. ,n one 
embodnnen,. ^e first CMPS arrangement to recetve content might govern the control 
process by downloadmg a„ .mtial CMPO and/or other CI. and display one or more of the 
m es to the user, etc. The second CMPS arrangement m.ght tecognize that ,, bas no tunher 
role ,„ play, enber as a result of a communication between the two CMPS arrangemenis or 
as a resu t of changes to the content stream created b, the first CMPS atrangem „. (wh * 
decrypted the content, and may have allowed demuxtng, composition and rendering e^, 

comnh 'TT' """"^'"^ ™''^^ '"-8-™^ ™y be 

cm heated if one device handles certatn aspects of MPEG-4 rendering, and the other 

and es oiber aspects. For example, a DVD player might handle demuxing and buffering 

.ransfetrtng raw ESs to a digttal TV, which then handles composition and rendenu. as 

we as display^ 1„ such a case, there m.ght be no back-channel from the composit,:; and 

^ndertng block to the upstream CMPS arrangement. CMPS arr^gemen. are preferably 

designed to handle stand-alone cases ,a DVD ,„r any other optical disk, player wttb a 

CMPS arrangement attached to a dumb TV with no CMPS), multiple CMPS arrangement 
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cases in which one CMPS arrangement handles all nfth. 

. . J- , . '"males all ot the processing (a DVD for other 

optical disk) player which handles evervthins throuph • • 

■ . cverytnmg through composition and rendering with a 

video stream output to the digital TV cir, ^ ■• . 

senal bus^ rth.r / non-hmiting example, via an IEEE 1 349 

senal bus) (that output stream would be encrvoted as ner th. -r 

P^"^^"^ "ve company proDosal" for 

2- Initialization of a particular content .tr..». 

The CMPS may be designed so that it can accept initialization information 

ma be a CMPO and/or other CI, may contain infonnation used by the CMPS to locate 
and/or mterpret a particular content stream as well as n n.c. /a ■ u , 
initi;,! h..A u associated with that stream. This 

.nit al header may be received through a sideband channel, or may be received as a CI ES 
such as a CMPO ES. '-civea as a ci bb 

InoneexamplcshowninFIG. 26, Header CMPr>9^;ni ■ , . 
information: "^'^ '"^'"^^ ^'^^ f«"«^i"g 

(a) Stream/Object/CMPO ID 2602, which identifies the content 
streams/objects governed by Header CMPO 260, and/or identification of CMPOs 
associated with each such content stream or object 

rules and r "'^'^^ '^'''^ ' ''''''''' ^^^^^ ^^^^^ -tain 

.le an^k.^ -^er embodiment. Header 

associated with such streams. In the latter case, no other CMPOs mav be used 

mcmp:~::::""^""""^'-^^^ 

(b) One or CMPO Keys 2603 for decrypting each identified CMPO 

.^^'^^ ^''^'^-L-^' Control 2604. consisting of basic control infonnation 
associated with the work as a whole, and therefore potentially applicable to all of the 
ontent streams which make up the work. This basic control information may include rules 
governing the work as a whole, including options to be presented to the user 

(d) In one embodiment of this embodiment, a header CMPO may be 
updatable to contain User/Site Informatinn i^n^ 

authori..H f ■ 2605 regarding a particular user or site currently 

authorized to use certain content as well a« r,„» ^ . 

. more rule sets under which the user has 

gained such authorization A heaH<>r TMPn ... 

^ . ,. " ^^^^ associated with a work currentlv beine viewed 

may bestoredinRAMorXfVRAN/i tu;. ' "s 

KAM or NVRAM. This may include updated information. In one 
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embodiment, the CMPO may also store header CMPO, fn ^ • 

, , ^^"^ for certain works v ewed in the 

past. In one embodiment, header CMPOs may be stored in nn 
■ r ■ Stored in non-secure memorv with 

(a) The header CMPO is received by a CMPS arrangemem^ In ,he case of 

n„.v be.„g used (e.g., a seMop hox wi* 500 cha™e,s, of wh.ch eiU,er 0 or J be,n. 
^.Played a, .y given ..™e,, CCMPOs fo, each channel .ay he buffered hv le CMPS 
ar^angen^en. for poss.hle use ,f ,he user invokes parircuiar conren. (e.g.. swi.hes ,0 a 
particular channel). s . iwiicnes to a 

In ei,her case, Ure header CMPO .us, include infonnadon wh,ch allocs a CMPS 
anangement 10 Identify i, as a header CMPO. 

clear ,„ ,h. h T IT,^""^ """"^ ""^'"^-model ,„fonna,io„ held in the 

ear ,„ the header CMPO. Business-.odel informanon may .nclude, for example a 
.a e.en, ^ ,f ,dver,.se.c„,s are included, r f h user 

e ir ■"'°™"™' -asurenten. .nflmrafot r 

-antple, content may he output to a server or other^se copied once, hut only at a pr.ce 

hasauthn H '™^""'"'™''"^'=«P'^"'«l>usinessmodel.iftheuser 

CMPS arrangement to always accept play w,th advernsements for ,>ee,. rejects the 
us,ness model, „the user has .nstructed that the particular model always e rejected or 
^.splays the business nrode, to the user (e.g.. hy presen.tng opttons on the screel, 

decrvnrs ,h "«= "''''^ """Semen. Uren 

„ Z ""^ APP'-" -ai- a live 

™ PU t cormectton to an e.emal server (e.g., Internet conneCon, baCch^e, on a set-op 
box, etc.,, and ,f latency problems are handled, deception of these keys can be handled by 
c^nrmuntcatmg with the extetnal server, each side authent.cating the other, estabitshm n 

secure ch^el, and rece.pt of a key from the server. ,f the Commerce Applian^rL 
t east oceastonally cotmected to an external server, decryptton may Have to bTbased on 
one or more keys securely stored in the Commerce Appliance. 

(e) Once a header CMPO has been decrypted, the CMPS arrangement 

acquires inloimation used to identify and loca,, ,h . 

ueniiiy ana locale the streams contammg the content, and 
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keys which are used to decrypt either the PMPHc aco^ • , j u > 

ypi ciiner me CMFOs associated with the content, or to directly 

decrypt the content itself. 

(f) In one embodiment of this header embodiment, the header CMPO may 
contam a data structure for the storage of information added by the CMPS arrangement. 
Such information may include the following: 

( 1 ) Identification of user and/or Commerce Appliance and/or CMPS 
arrangement. In this embodiment, such information may be stored ,n a header CMPO in 
order to provide an audit trail in the event the work (including the header CMPO) is 
transferred ( this only works if the header CMPO is transferred in a writable form) Such 
information may be used to allow a user to transfer the work to other Commerce 
Appliances owned by the user without the payment of additional cost, if such transfers are 
allowed by rule information associated with the header CMPO. For example, a user may 
have a subscription to a particular cable service, paid for in advance by the user VVTien a 
CMPS arrangement downloads a header CMPO from that cable service, the CMPS 
arrangement may store the user's identification in the header CMPO The CMPS 
arrangemem may then require that the updated header CMPO be included if the content is 
copied or transferred. The header CMPO could include a rule stating that, once the user 
information has been filled m, the associated content can only be viewed by that user 
and/or by Commerce Appliances associated with that user. This would allow the user to 
make multiple copies of the work, and to display the work on multiple Commerce 
Appliances, but those copies could not be displayed or used by non-authorized users and/or 
on non-authorized Commerce Appliances. The header CMPO might also include a rule 
stating that the user information can only be changed by an authorized user (e g if user I 
tninsfers the work to user 2, user 2. CMPS arrangement can update the user information m 
the header CMPO, thereby allowing user 2 to view the work, but only if user 2 is also a 
subscriber to the cable channel). 

(2) Identification ofparticular rules options goveminc use Rule 
sets included in header CMPOs may include options. In certain cases, exercise of a 
particular option might preclude later exercise of a different option. For example a user 
might be given the choice to view an unchanged work for one price, or to change a work 
and view the changed work for a higher price. Once the user decides to change the work 
and view the changed work, this choice is preferably stored m the header CMPO since the 
option of viewing the ong.nal unchanged work at the lower price is no longer available 
The user might have further acquired the right, or may now be presented with the option for 
the right, to lunher distribute the changed work at a mark-up in cost resulting in third party 



wo 99/48296 

PCT/US99/05734 

-58- 

derived revenue and usage information flowing to i^nth th. 
stakeholder(s). ^ ^ ''^^ 

U^ge i„f„™a.,o„ „,ay „,ed ,o de.e™„„e if addi,i™a, us« are au.honzed bv 

h,„ „ , , . '''™f> »«>* ™y be viewed for free 

6u. only ,f h.sioncal usage i„f„™a,i„n is downloaded ,o a server 

Conlenl Managemenl Proleclion Object, (CiMPO) 

which r ■ " — 

ma, e fo™a„ed as a dau s.r„e,are specified b, a par„en,ar srandard (e, . a„ MPEoT 

...hzed by ,he srandard ,e.g , as par. of a composite MPEG-4 srrcan,, or nray be received 
■hrough some other, side-band method If the CMPn f . may ce received 
SDecilledhv.l,, , "'"^'-'^^'^fo'maitedasadatastrucnirenot 

.1 tZ '^"'^* -^""'^ - - - — ™. 

iiiLiuae receipt through a separate port. 

levels r"'" ^' ^'"-"'^ ™^ ^--'-^ ^« =-mpla,y 

levels w,ll be dtscussed herein: "channel,- "work,- and "object " 

C o by ,he user (e.g., a web site, or a vtdeo libra,,, or trray be ..ce.ved senally (c g a 
cable televiston channel). «ria,iy |c.g., a 

A "work" represents a stngle audio-visual, textual or other work, .ntended to be 
consumed (viewed, read, etc., by a user as an integrated whole, A work may f ^.e 
= a mo„e. song, a magaztne arttcle, a multimedia product such, for example as 
soptusttcated videogame, A work may .ncorporate other works, a., for e Jpi:Tn a 
™,„med,a work which tncorporatcs songs, video, te,t, etc. ,„ such a case, rL may be 
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associated 

An "object" represents a separately addressable portion of a work. An object n,av 
be. tor exatnple, an individual MPEG-4 AVO, a scene descnptor graph, an object ' 
escnptor, the soundtrack for a n,ov,e, a weapon .n a videogame, or any other log.callv 

definable portion. ^ ' 

Con,e„, may ba controllM a, any of .hese levels (as well as in,ermed,a,e levels no. 
discussed he,e,„)^ The preferred e.bcd.„.e„. mechanism for s.ch con.ro, is a CMPO or 
CVIPO an^gemen, (which con,prise3 one or more CMPOs, and .f plural, then plural 
cooperaSng CMPOs). CMPOs and CMPO a,ra„ge.en,s „ay he organized hierarchically 
w,^ a Channel CMPO an^genren. imposing rules applicable ,o all conrained works a 
MCMPO or an SOCMPO ..posing ™,es applicable ,o all objecis wHhin a worl.: and a 
CMPO arrangement imposing rules applicable to a panicular object 

CrJnZ " ^"''^ "O"""-" CCMPO 2701 . 

CCMPO 2701 may include one or more Rules 2702 applicable to all content in the 

channel as well as one or more Keys 2703 used for decryption of one or more MCMPOs 

ando, SOCMPOs. MCMPO 2704 may include Rules 2705 applicable to a single woT 

and/or worts, one or more classes and/or more users and/or use, classes, and may also 

include Keys 2706 used ,o decrypt CMPOs. CMPO 2707 may include Rules 2708 

applicable to an individual object, as well as Key 2709 used to decrypt the obiec, 

As long as all objects are subject to control at some level, there ,s no rc<,u.remen, 

Iha, each obiect be individually controlled. For example, CCMPO 2701 could specity a 

s.n.^ mle for viewing contem contained in „s chamtel (e.g., content can only be viewed by 

a subscriber, who is then migh, be ftee to redistribute the content with no lurlher obligation 

0 the content provider,. In such a case, rules would not necessarily be used for MCMPOs 

(e g. Rules 2705), SGCMPOs, or CMPOs (eii Rule, 97081 , 

^^^i^^i> Ve.g., Kuies 2708). In one embodiment 

MCMPOs, SGCMPOs. and CMPOs could be d.spensed w.th, and CCMPO 270, could 
■nclude all keys used to decrypt all content, or could specify a locatton where such keys 
could e located. In another entbod..en., CCMPO 270, would supply Key 2703 used to 
decrypt MCMPO 2704. MCMPO 2704 nught include keys used to decrvpt CMPOs (e . 
Keys 2706), but n^ight include no add.t.onal Rules 2705. CMPO 2707 might include 
-709 used to decrypt an object, but might include no additional Rules 2708. In certam " 
embodiments, there may be no SGCMPOs. 

A CMPO may be contained within a content data structure specified by a relevant 
standard (e.g., the CMPO may be part of a header in an MPEG-4 ES.) A CMPO may be 
contamed wthm us owt.. dedicated data structure specified by a relevant standard (e'o a 
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CMPOES). ACMPOn.aybecon,ai„edv.thi„adatastructurenotspec,f,ecibyany 
content standard (e.g., a CMPO contained within a DigiBox). 
A CCMPO may include the following elements: 

(a) ID 2710. This may take the following form: <chamiellD>< CMPO 
type><CMPO ID><ve..on number>. m the case of h.erarch.ca. CMPO organ.zation (e.g 
CCMPOs controlling MCMPOs controlling CMPOs), CMPO ID 271 1 can include one 
field for each level of the hierarchy, thereby allowing CMPO ID 271 1 to specify the 
location ofany particular CMPO in the organization. ID 2710 for a CCMPO mav for 
example, be 123-000-000. ID 2712 for a MCMPO of a work within that channel may for 
example, be 123-456-000, thereby allowing the specification of 1 ,000 MCMPOs as 
controlled by the CCMPO identified as "123." CMPO ID 271 , for a CMPO associated 
with an object within the particular work may, for example, be 1 23-456-789 thereby 
allowing the specification of 1 .000 CMPOs as associated with each MCMPO 
rx.o. °f ^P-.fying CMPO IDs thereby conveys the exact location of any 

CMPO within a hierarchy of CMPOs. For cases in which higher levels of the hierarchy do 
not exist (e.g.. a MCMPO with no associated CCMPO), the digits associated with that level 
ot the hierarchy may be specified as zeroes. 

(b) Rules 2702 applicable to all content in the channel. These may be self- 
contained rules, or may be pointers to rules obtainable elsewhere. Rules are optional at this 

level. 

(c) Information 271 3 designed for display in the evem the user is unable to 
comply with the oiles (e.g., an advertisement screen informing the user that a subscription 
IS available at a certam cost, and including a list of contem available on the chamiel). 

(d) Keys 2703 for the decryption of each MCMPO controlled by this 
CCMPO. In one embodiment, the CCMPO includes one or more keys which decrypt all 
MCMPOs. In an alternate embodiment, the CCMPO includes one or more specific keys 
for each MCMPO. 

(e) A specification of a CMPS Type (2714), or of hardware/software 
necessary or desirable to use the content associated with this channel. 

The contents of a MCMPO may be similar to those of a CCMPO. except that the 
MCMPO may include rules applicable to a single work, and may identify CMPOs 
associated with each object. 

The contems of each CMPO may be similar to those of the MCMPO. except that 
the CMPO may include rules and keys applicable to a single object. 
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.he MCMPO .ay ,„cMe n.,es appUca* ,o only c=m,„ one o, ™o,. dasses of righu, 
ce„.,n one or More classes of works, and/or ,„ one or „ore ce„a,„ classes of users and/or 
user arransemcnls (e g. CMPO arrangemen,s and/o, Iheir devices). 

'"''"<"''"™''Odimcn,.showninFIG.2S,CMPODataS.ruc,nre280l maybe 

defined as follows: 

CMPO Data Slmcture 2801 is made up of elements. Each eiemen, includes a self- 
conramed i,em of informanon. The CMPS parses CMPO Data Slrucrure, one elemen, a, a 

time. 

ruP. ''T "^'^ ^'""""^ " ' "--"y .He 

CMPS ,0 d,s.,„gu,sh ,, from , conren, ES. ,n an exemplary embodrmen, Uns elemen. may 

mcludj 4 bus, each of which may be se, ,o " 1 - ,o indicaie ,he data srnrcure ,s a 

The second elemen, is CMPO Identifier 2803, which is used to identify this 
particular CMPO and to convey whether th^ r\Avr, ■ _ x- , ■ 

pwp^ ^ , ^ ^^PO P^rt of a hierarchical organization of 

CMPOs and, ,f so, where this CMPO fits into that organization 

CMPO Identifier 2803 is divided into four sub-elements, each of three bits These 
are sho.vn as sub-elements A. B. C and D. The first sub-e.ement (2803 A) identifies the 
CMPO type, and indicates whether the CMPO is governed or controlled by any other 

100: this is a top-level CMPO (associated with a channel or an aggregaUon of 
works) and is not conu-olled by any other CMPO. 

010: this ,s a mid-level CMPO (associated with a particular work) and is not 
controlled by any other CMPO. 

110: this is a mid-level CMPO, and is controlled by a top-level CMPO 

001 : this is a low-level CMPO (associated with an object wuhm a work) and is not 

controlled by any other CMPO. This case will be rare, since a low-level CMPO will 

ordinarily be controlled by at least one higher-level CMPO. 

Oil: this is a low-level CMPO, and is controlled by a mid-level CMPO, but not by 
a top-level CMPO. ^ 

111: this is a low-level CMPO, and is controlled by a top-level CMPO and by a 
mid-level CMPO. 

r..ur. (sub-element B) identifies a top-level 

CMPO. In the case of a top-level CMPO, this identifier is assigned by the creator of the 
CMPO. In the case of a mid-level or low-level CMPO which is controlled by a top-level 
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uch co„«,L ,„ ,he case of a „,d-levcl „, low-level CMPO which is no, comrolled by a 
loivlevel CMPO, this sub-elemem co„u,i„s zeroes^ ' 

CMPO '° ^» ^ 

CMPO, ,„ ,hc case 0, a ,„p,evel CMPO. ,his sub-eleMen. contains .eroes. In ,he case of 

= ™d.,eve CMPO. ^is su^,e.e„, co„«i„s .he ,.e„,.flca,io„ of .e p.„,c„,a. CMPO „ 

.he case o a „w-,eve, CMPO wh.ch is co„.o„ed by a n-id-ievel CMPO. .his sub-e.e J, 

co„.a.„s ^e ,den.i„ca„o„ of .he ™,d-,eve, CMPO which pe.fon.3 such c„„„„,. ,„ J 

co:i::::r'™""''"^"°'""""'"^-"---™--'----". 

CiMPO. h .he case of a ,op.,evel or .,d-,evel CMPO, .his s„b-elemen. coma.ns zeroes 
In ^e case 0, a low-leve, CMPO. ,h,s sub.,c.e„, con.,ns .he ,de„,.fica„o„ of ,he 

particular CMPO. 

CMPO d°,"°"'"' ■"^-""^ '^^ »' -He 

e e" en. ; irr^- ""^ '""'""^ ""-^'^ » fina, 

been al?d ! ™' '^^ 
een a,.ered w„h„u, pe™ission. s.nce such an ai.era.ion „,gh, ,.su„ i„ a d.fferen, size 

r„. ced da.abase. This i„fo™a.i„„ can ^ ,„ 
has been .eccved and is ava.iable, pno, .o any a„en,p, ,o proceed wi.h process.; 
F„I,ow,ng Size Elc„,cn. 2804 are one or „,ore Ownersh,p/C„n.,ol EI,n,en,s 

28 X 2806 and 2807, ,n .he nrs. such =,en,en, (2805), .he crea.or of ,he CMPO .a, 
.nclude a specinc iden.,f,cr .ssocia.ed with .ha, crea.or. Add,.ional pariicpan.s .ay'also 

:r ' — p'=- ^^0' 

.den.,^ .he crea.or of ,he CMPO. Elenren. 2806 could ,de„.i,V -he publisher of ,he 
assoc,a.ed work and Elemen. 2807 could idc„.ilV , he auU,or of d,e work 

A spccfic End Etoem 2808 sequence (e.g.. 0000, i„dica.cs .he end of .he cha,„ of 
wnershrp e,e.e„,3. If ,.3 sequence ,s encoun,ered ,n ,he „rs. e,.n,e„,, .h,s ,„d,ca,es .ha, 
no Cham of ownership infontlarion is presen.. 

Chain of ow™rship informaiion can be added, if rules associaicd w„h CMPO 2801 
pe™,. such addi.,ons 1, for example, a user purchases ,he work ass„c,a,ed wnh CMPO 
-801. ,he users ,de„.,fica„o„ may be added as a new elemen. .„ d,e chain of o.vnersh.p 
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elements (e.g., a new element following 2807 but before 7«nR^ tu- 

2809 T "r""" " "-^'-^ 

any CMPS which has downloaded and decoded CMPO 2801 .nH/ 

■,A .-i- ■ „ '^'^ and/or may conta n the 

ee„ c„cu,a,e. tap„p.H, Such ,„f„™a„„„ .a, a,so ,eponed a. exha J, a 
cl=an„g.„u3e „. cental se„e. Cha,„ of Ha„„i„, .„fc^,„„„ _ ; 

~ .pon... ,f „e „„..„„f e,e.e„,s fo. .„ch .nfLa^l.eeds a 

specned amount (e.g., twer^ separate user idenlffiers) a CMP^ , 

.«.»p™ce.„,.fCMPO«„.,t.ea3.ooJ:r„t:^^^^^^^ 

— .oa^e„e™a,.„e.a„dHa„epo„edt.cha,„ofh.d,,„,.f„:I: 

The last element in the chain of handling elements 9«, n • ^- 
...o fe,e , 

(e g ,8 : - - = Be.e„ts 

u h , " " "^^o^'a'ecl w..h this CMPO 

such a d,g,.a, ce„,flca,e .ay be used by ,he CMPS ,o au,he„,.ca,e the CMPO T e fl 

le.c„, ,„ the digua, ce„.„ea.e cha.n ,s a„ zeroes ,28,4,. ,f „„ d„ta, certihca. 
present, a stngle elentem of all zeroes exists in ,h,s l„ca„o„ 

-81..-8.6^28I7,28.8)spec,.yi„g one or tnore content objects and/orCMPOs which Jv 

found <e.g., these may be stored ,n locations 2815 and 2817,. Following each such 
.denttfier .ay be one or .ore keys used to decrypt such CMPO or object ,e g Id ,„ 

■ocattons .8,6 and 28 1 8,, The se, Of ident,„er*ys ends With a terr^ina, l:^^^^^^^^ 
made up of all zeroes (2819). ^'cmcni 

Following the set of elements specfying ide„,if,crs and/or keys may be a se, of 

Rules Elements (e R 2820 - ^ ^ a^ciui 

with use of the , : ™d conditions associated 

wtth t« of the content objects and/or CMPOs tdentified in the Governed Objects chain 

(e.g., locations 2815 and "»S 17^ rv,.r„„io . '^"jccis cnain 

and .817). E.xemplar>' rules are described below. Elements may 
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of *e .social c.„,e„, ob,ec,s. ^ " ~ - 

Element 2823 coma,„,„g ,„,„™a„„„ .peeffied by ,he c,ea,„, of U,e CMPO A . 
conKms, such infomalion mav include coment or n„- , ^"^ 

The CMPO ends wilh Fmal Tem,i„a,i„„ Elemen, 2824 

-..:r:::;:::::hrro::r^^^^^ 

COP.V io tj: J'' "'""^ '» — <--^ ™. .« 

'elevan, hudge.s, a. no, v,o,a,ed or e eld TcmTs I" ' " '""'"^'"^ 
-a. a Cop, „pe.,„„ .a, cause an upda.e o an assi erM^ "'^ T^'"" 
indicadon Iha, the associa.ed con.em has bee T! ' ' ™ 

.he sue responsrble for „,ak,n Th ' '""^'""^ 

comen, obiec, an '"^8= "> ™V applicable 

oment objec and ,n pan.cular wuhou, requinng d,a, assocaied con.en, objecrs be 
demuxed, decrypred or decompressed. ,„ ,he case of MPEG-4 for exa™„t h 
'"""^"B nruUr-srage denrux process- ' ' ' 

-.orfron.aheadercL;^"''™"'""''"'^^^"'^-'"---''- 

(ii) CMPO ESs associated with the MPPn Ac, \- , 
copied are separared fro. ,he con.en, s,ream In a «rs. d^ ux s!,. ^ " 

-.cMPosa.e.henr::::er::ernrL::r:r^"™"^-"-- 

-o.^X....e.,res.rea„,srou.ed.::rp::ru:^^^^ 

»n.en.:~:d:~ 

ana decrypted. It requires that the CMPS arrangement include 
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two outputs: one output connected to the digital output port (e.g., FIG. 23 line 2316 
connect g. Dig,,, Output Pon 23 1 7). .d one output connected to the MPEG-4 buffers 
(e-g., FIG. 23, hnes 23 1 0, 23 1 23 1 2), w.th a switch designed to send content to one 
output or the other (or to both, if content is to be v.wed a.d copied simultaneously) (e . 
Sw,tch 2319). Swtch 2319 can be the only path to D,gUal Output Port 2317. thereby ^ ^ 
allowing CMPS 2302 to exerc.se direct control over that port, and to ensure that content . 
never sent to that port unless authorized by a control. If Digital Output Port 231 7 is also 
U.e connector to a d.g.tal d.splay device, CMPS 2302 wHl also have to authonze content to 
be sent to that port even if no copy operation has been authorized 

In one example embodiment, the recetvmg device receiving the information 
through D,ghal Output Pon 23 1 7 may have to authenticate w,th the sending dev.ce (e g 
CMPS 2302). Authenfcanon may be for .y characteristic of the dev.ce and/or one or 
more CMPSs used in conjunction with that dev.ce. Thus, for example, a sendmg appHance 
may not transma content to a storage device lackmg a compatible CMPS 

In another non-limitmg example, CMPS 2302 can incorporate session encryption 
unctionality (e.g., the "five company arrangement" ) wh.ch establishes a secure channel 
from a sendmg mterface to one or more externa, device interfaces (e.g., a digits monitor), 
and provided that the receiving interface has authenticated with the sending mterface 
encrypts the content so that it can only be decrypted by one or more authenticated 13^4 
device interfaces. In that case, CMPS 2302 would check for a suitable IEEE 1 394 senal 
bus mterface , and would allow contem to flow to Digital Output Port 23 1 7 on.v if (a) an 
authorized Play operation has been invoked, a secure channel has been established with the 
device atid the content has been session-encrypted, or (b) an authonzed Copy or Retransmit 
operation has been invoked, and the content has been treated as per the above description 
O.e., the CMPO has been demuxed. changed and remuxed. the content has never been 
decrypted or demuxed). 

This is only possible if CMPOs are separately identifiable at an early demux stage 
which most likely requires that they be stored in separate CMPO ESs. If the CMPOs are ' 
stored as headers in content ESs, it may be impossible to identify the CMPOs prior to a 1^1. 
demux and decrypt operation on the entirety of the stream. 

(4) Change. The user may be authorized to change the content. 

(5) Delete. This command allows the user to delete content which is stored 

•n the memory of the Consumer Appliar.ce. This operation operates on the entire work If 

the user wishes to delete a portion of a wnrt tu^ nu 

a ponion oi a work, the Change operation must be used. 

(6) Transfer. A user may be authorized to transfer a work to a third party. 
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This dife fe„ ^ copy „p..„o„ ,„ ,ha, U,e user does no, re,ai„ U,e co„.e„. or an, 
nsh.s ,o the eonren,. The Tr^sfer opera„o„ „ay be ca.ied o„, ,y co™«n a Z 
operauon and a De,e,e operation. Transfernra, re,u,re a,.era„on f ,he hea r cmTo 
associated With the work s addinanr u ■ ^ "caaerLMFU 
n, °' "''^""8 ^ Ownership/Control Element such as 

Elements 2805-2807 of Fir, 98^ '>='"ciu, sucn as 

^« - associate nghts to the work with the third party 

These basic operations mav be subiert tn ,T,^w-r . , 
^ ^ ' ^ ^''*'J^^^ *o modifications, which may include 

, , ' °P^^^»'°"s '"ay be conditioned on some tvpe of user 

pa™. Payment can take thefonn of cash payment toaprovider(e.g.,credicr^^^^^ 
sub.act.on from a budget), or sending specified infonnat.on to an exteL sit f 
Nielson-type information). * ' 

Quality of Service. Operations may specify parricularouahtv of 

lov 1 of decompresston, re„uested/requ,red types of display, rendenng deviees'te . In.he 
9ual.ty loudspeakers, a particular type of game eonuoller, ' ' ' 

on..lo»eda«erapal::i:~^^^^^^^ 

time (e = real tim. f operation is tied to the 

.ne , .g., real-ttme information at a price, delayed tnfo^ation at a lower price or free 

e.g.. allowing controlled copies bu, only after a particular date). 

condi,- . . °''*''"P'«"^''l"'yi«'^°fcomen.. Operationsmavbe 

be free If the user agrees to allow advertisements to be displayed) 

in all of these cases, a rule may be modified by one or more other rules A rule mav 
jcly tha, ,t can be modiDed by other ™,cs or may spec.ly that ,t is unmodifiab e a 
™ie IS mo ifiable, i, may be modmed by rules sen. from other sources. Those rules lay 

the folC * - ^"^-^ 

a. CMP Data Stream. 

The CMP-ds is a new elemental stream type that has all of the properties of an 
element^ stre^ including its own CMPO and a reference m the ob.ct deLpior Lb 
CMP-ds stream has a series of one or more CMP Messages. A CMP_Message ha. four 



1- Count: [l...„J CMPS types supported by this IP ES. Multiple CMPS 
systems may be supported, each identified by a unique ^-pe. (There may have 
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to be a centra] registry of types. ) 

2. CMPS.typeJdentiHers: [!....] identifiers, each with an offset .n the 
stream and a length. The offset potnts to the byte in the CMPO where the data 
or that CMPS type is found. The length ts the length in bytes of this data 
. Data segments: One segment for each of the „ CMPS types encoded in a 
format that is proprietary to the CMPS supplier. 

4. CMP_Message_URL: That references another CMP Message. (This is in 
keepmg with the standard of using URLs to pomt to streams ) 
b. CMPO. 

The CMPO is a data structure used to attach detailed CMP control to individual 
elementary streams. Each CMPO contains: 

1. CMPO_lD: An identifier for the content under control. This identifier must 
uniquely identify an elementary stream. 

2. CMPO_count: [\...n] CMPS types supported by this CMPO 

3. CMPS_typeJdentifiers: [l --] identifiers, each with an offset m the 
stream and a length. The offset po.nts to the byte in the CMPO where the data 
for that CMPS type is found. The length is the length in bytes of this data 

4. Data segments: « data segments. Each data segment is m a format that .s 
proprietary to the CMPS supplier. 

5. CMPO^URL: An optional URL that references an additional CMPO that 
adds information to the information in this CMPO. (This is a way of 
dynamically adding support for new CMPSs.) 

c. Feedback Event 

'^he feedback events come in two fonns: Start and end Each feedback event 
contains three pieces of information: 

1. Elementary stream lD 

2. Time: in presentation time 
3- Object instance number 

User Interface. 

Co^nerce Appliance 230, may .nclude Use, Imerface 2304 des.gned to convey 
con„^.,cla,ed i„f„™a„„„ ,„ „e and ,o ,ece,v. co™,a„ds and i„f„n„a„o„ fro™ d,e 
user. Th,s ,„,erface may include special purpose displays (e.g., a ligh. which comes on ,f a 
c™ acuon requires paymen,). spec.al purpose bu„ons (e.g., a bunon wh.ch accep,s the 
payment or o.her ,en.s re,u,red for display of conren.,, a„d/o, visual infor^auon p.se„,cd 
on screen. 
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Example of Operation in an MPEG-4 Context 

1. User selects a particular work or channel ru 

only be vewed by .ubscnb... and ™av ' 

MCMPO. ° ^""^ " — ^'^^ -ed f„. dec^p„„„ „f 

5 The CMPS an-angeraemdovvnloads the MCMPO Inih,. t . 
embodimenl, ,he MCMPO may be on PI, o °' 

.hose .lTZZZ7r' ''T " ^ 

™geme„, pa^es d,e scene d The CMPS 

o.ber seene dl^X;:; '^'"'-"'-•^ ^'-^^ 

cMPsa™,e™e„.dej::: ::rrr"r""""''^^ 

temmales fanher decryption 
CMPS Ri^,, ^.„„^.„.„, ^^^^^^ 

add,t,o„ ,0 co„s„„,e, a™„ge.en„ ;„ ^^^^ 
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may b. used i„ creating, cap,„,i„g, „,„,i^,„,, 

X racng ..,,di„, e^..„, .„^„,„^, „„^^^^,_„^ ^^^J^ 

'";"'"°"-l'"'i'i»i"<™Pl=.aCMPS,a„o„.exha„s>iveexampleofwh,chmav ' 
.nCude a .as, a .ecu. p„„,„„ „f , VOE node a. de.cn W in U,e af„L„„„„l" 1, e. 
a... pa.e„, spec,f,ca,o„. ,s i„eo,po,a,ed i„ video and digi,a, ca„,er.. audio . Jp" ^ 
-ord,ng Playback, ed„i„g, ^d,o, „o.se ,educU„„ dev,ces and/„. „y o^er d,g..a del 
l™.es, v,deo, a.d/0. audio, or any „,Ker re,eva„. digi., i„fo™io„ Ly be ca u^^ 
rear ed, »d pe„i.e„„y p„,ec,ed using a. ,ea. one CMPS and.or a, ,eL „„e ^^0 
CMPS,™ay,„,e,ac,wi,hco™pressio„/decomp,«,o„,cnc,yp,ion/dec,yp„o„. DSP digiu, 

ediiin. 'c -imauon. spec.a, effec. d,gi,a, 

d.e,.a, ,nfo™a„o„ „ay pro,=c, and/or manage rights associaied w„h dig„al i„f„™„ion 
usmg at leas, one CMPS and/or at leas, one CMPO. '"lonnauon 

assets »f CMPSs and/or CMPOs to manage digttal 

,n a, leas, one digtta, l.bra^, asse, s,ore. a„d,or audio libraries, di tta, aults 
and/„r any other dtgital content storage and management means 

In accordance with the present applications, CMPSs and/or CMPOs may be used ,o 
.»a.e „gh,s .„ comunctton w„h the public dtsplay and/or perrormance ordigL ^ 
'n one non-exhausuve example, flat parrel screens, d.splays, monitors. TV proctors LCD 

leas ne «„ar. and/or software CMPS instance that controls the use of d.gital worKs A 

of whtch ,s a dtgnal certtncate, that warrant that use of the drg.tal tnformation will occur n 

»,p.es Of sa,d contexts mclude ,heate„, bars, clubs, electrome billbo^ds, electronic 
dtsp tn pub c areas, o, TVs ,n airplanes, ships, trains and/or other public conveyances 
These credenttals may be issued by tested tWrd parties such as certtty.ng authoHties notl' 
exhausttve examples of which are disclosed ,n the aforementtoned Gi'nte^Tia pa. ^ 

application. ^ 
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Additional MPEG-4 Embodiment Information 

J^'' ^-^^ - ^^-^ - the MPEG-4 description in the vers.on 1 Systems Committee 
Draft (CD), currently the most complete descnption of the evolving MPEG-4 standard. 

This section presents the structural modifications to the MPEG-4 player architecture 
and discusses the data lines and the concomitant functional changes. Figure 23 shows the 
fimctional components of the original MPEG-4 player. Content arrives at Player ^301 
packaged into a serial stream (e.g., MPEG-4 Bit Stream 2314). It is demultiple.xed via a 
sequence of three demultiplexing stages (e.g., Demux 2305) into elementary streams. 
There are three principle types of elementary streams: AV Objects (AVO) Scene 
Descriptor Graph (SDG), and Object Descriptor (OD). These streams are fed into 
respective processing elements (e.g., AVO Decode 2307, Scene Descriptor Graph ^306 
Object Descriptors 2308). The AVOs are the multimedia content streams such as audio' 
video, synthetic graphics and so on. They are processed bv the player's 
compression/coding subsystems. The scene descriptor graph stream is used to buUd the 
scene descriptor graph. This tells Composite and Render 2309 how to construct the scene 
and can be thought of as the "script." The object descriptors contain descnption information 
about the AVOs and the SD-graph updates. 

To accommodate a CMPS (e.g., CMPS 2302) and to protect content effectively the 
player strucnire must be modified in several ways: 

• Certain data paths must be rerouted to and from the CMPS 

• Certain buffers in the SDG, AVO decode and Object descriptor modules must 
be secured 

. Feedback paths from the user and the composite and render units to the CMPS 
must be added 

In order for CMPS 2302 to communicate with the MPEG-4 unit, and for ,t to 
effectively manage content we must specify the CMPO structure and association protocols 
and we must define the communication protocols over the feedback systems (from the 
compositor and the user.) 

The structural modifications to the player are shown in Figure 23. The principal 
changes are: 

• All elementary streams are now routed through CMPS 2302. 

• Direct communication path between Demux 2305 and CMPS 2302. 

• A required "Content Release and Decrypt" Module 23 1 5 in CMPS 2302. 
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• The addition ofa feedback looD fee f in*. OTi -!^ fi- 

2309 .oCMPS 2302. ^™ -^") Co.p„s„e and Render 

• Bi.di,ec.,o„al user i„,erac.,„„ dnecly ™u, ,he CMPS 2302. .hrough Line 23 1 6 

s^an,s Elen,e„,a„ s„ean,s ,ha. the au.ho, chooses „o, ,o p,„,ec, are s.ill nrarked hy an ' 
>mpro,ec,ed conrem" CMPO. The CMPOs are ,he pri^ar, means of an^ching rules 
mfomrauon ,„ ,he conren,. Cornea, her. no, o,.y refers .o AVOs, bu, also ,0 ,h. scene 
desc„p,or graph. Scene Descnp,or Graph ™a, have g,.a, v^„e and wiU ,h„s need ,o he 

protected and managed by CMPS 2302. 

T;^'' ^^'P^I-fron-De".-" 2305,0 CMPS 2302 ,s used «, pass a CMPS specific 
Kea^r, .a, p„,en,ia„y cousins business .ode, ,„fo™a,io„, .ha. co..u„,ea,es hu in 
^ode ,„f„nna„o„ a, .he heg,„n,ng of usersess.on. Uis header can be used ,o i„,„a.e user 
*n ..canon ^d au,hen.,ca,ion, co„.n.u„ica,e ru.es and consequences, .d .„.,a.e up. 
fron, ,„.erac.,o„ wi,h ,he rules (selecon of quahry-of-service (QoS), billing, e.c , The 
user s comn,unica,io„ wi,h CMPS 2302 is conduced .hrough a »0„.„.„<,..^,w channel 

■ ii,:: • ™" ™' - - - .-c 

pumos ' ' ""^^ - ~' 

purpose. The path .s used ,o cross check .ha, .he sys.c„ acually pr=se„,ed ,he user w„h a 

g.ven scene^ Elen,en.ary s.re^s .ha, a,, processed by .heir respec.ive modules may „„, 

«cker cou d pay once and view mul.iple ,imes. The .eedback pad, here allows CMPS 
.302 ,0 cross c eck ,he rendenng and thereby perfo™ a more accurarc accounung. Th.s 
feedback ,s .mplememed by forcing ,he Composue „d Render block 2309 ,„ issu'e a 
eje. .ha. s,gnals .he ,ni.,a„on of a given ob,ec, s rendenng ,ha, ,s complemen.ed bv 
.v™, upon «mnna,i„„. The feedback signahng process may be made opnonal b, ' 
provdrng a CMP-„„,.f.ca,.o„ flag ,ha, may be ,oggled ,o indica.e whe.her or no, CMPS 
2302 should be noUfied. All CMPOs would be „,uired ,o carr,- ,his nag 
AVO - •» 'ta ,he dear rex, buffers ,n ,he 

seL ^ ^~ Comp„s„e.and-Re„der block be 

secured. Th,s ,s .o preyen, a pirare from s.ealing comen, i„ U,e.,e buffers. As a pracucal 
™ner, ,h,s may be difficul,. since .ampenng wi.h U,ese s.n.cmres may well delrov 

r TT:"" ' -"K- -me from 

Placng ,hese buffers imo a pro,ec,ed processing environmem. 

CMPS 2302 ,o.err,s ,he ftmc.iomng of Player 2301. consis.en. wi,h .he .ollow,„g: 
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' cI™'''"'" mechanism between CMPS 2302 and the MPEG-4 player (via 

• A content release and decryption subsystem 

• Version authentication subsystem 

• Sufficient performance so as not to interfere with the stream processing m the 

MPEG-4 components 

CMPS 2302 may have a bi-directional side-channel that is external to the MPEG-4 
Playerthatmay also be used fortheexchangeof CMP information. Furthermore the 
CMPS designer may choose to provide a user interface API that provides the user with the 
abihty to communicate with the content and rights management side of the stream 
management (e.g., through Line 23 16). 

EnciTpted content is decrypted and released by CMPS 2302 as a ftmct.on of the 
rules associated w.th the protected content and the results of user interaction with CMPS 
-302. Unencr^^pted content is passed through CMPS 2302 and is governed by associated 
rules and user interaction with CMPS 2302. As a consequence of these rules and user 
interaction, CMPS 2302 may need to transact with the SDG and AVO coding modules 
(e.g., 23 10, 23 1 1 ) to change scene structure and/or the QoS grade 

Ultimately, the CMPS designer may choose to have CMPS 2302 generate audit trail 
mformation that may be sent to a clearinghouse authority via CMPS Side Chamiel Port 
^3 1 8 or as encrypted content that is packaged in the MPEG-4 bit stream 

The MPEG-4 vl Systems CD uses the term ^ object" loosely. In this document 
"object" IS used to specifically mean a data structure that flows from one or more of the 
data paths in Figure 23. 

Using multiple SD-graph update streams, each with its own CMPO allows an 
author to apply arbitrarily specific controls to the SD-graph. For example, each node in the 
SD-graph can be created or modified by a separate SD-graph update stream. Each of these 
streams will have a distinct CMPO and ID. Thus, the CMPS can release and decrypt the 
creation and modification of each node and receive feedback information for each node 
individually. The practical implications for controlling release and implementing 
consequences should be comparable to having a CMPO on each node of the SD-graph 
without the costs of having a CMPO on each SD-graph node. 

Principles consistent with the present invemion may be illustrated using the 

following examples: 

In the first example, there is a bilingual video with either an English or French 
soundtrack. The user can choose during playback to hear either the English or French. The 
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basic presentation costs Si If the French ^minHtr. v ■ 

ffth u presented there is a $0.50 surcharee 

Ifthe user switches back and forth between French =,nHP ru^ surcnarge. 

, . , ^'•English, during a single viewine of 

the presentation, the $0.50 surcharge will occur only once. 

In this example, there will be four elementary streams- 

i^piy a sToor ^^^^"'7 ^^'"^ ''-'^ ^ ^^'0- CMPO will 

-ply a S .00 fee associated with the use of the content. The scene description graph 
...ste^^^^^^ 

Fr nch. If the user clicks that button, the English stops, the French picks up from that point 
an the button changes to a switch-to-English button. (Optionally there ma^ be a 1.1. 

rtlTCpr"^ --^-asytodo 

The Video Stream with the TMPn ,1^11 *t_ 
, . "'^°"'"^!"'""'l"n only be released if the scene 

descnpiion graph upda.e stream above ,s released. 

The English Audio Stream will be similar to the Video stream 

charge IT'"" '"""^ '° ^'^^ - » 50 

change ,f ,s seen ,„ the feedback channel. (The CMPS must to no. count t^ce if the user 
swttche. between the ,wo in a single play of the presentatton ) 

the fe nt" " ""'""^ ^■^■8"""' "P"'"' ^-am appears in 

c feedback path (e Feedback Path 23 ,3, This is so CMPS 2302 knows whele 
presentatton stops and ends so that CMPS 2302 can co^ctly b.ll for the French audio 

vartattoL Ttrr'T ^ "^'^ '"^^^ """^^ 

seTth H f . ™^ --'"i-S "if don't 

=e the ,d lor the scene desertptton ,raph update stream X ,„ the feedback chamtel halt 

..o. be. Thts ,.es the vtdeo ,„ th,s one presentatton. Us.ng the video i„ some other 
presentauon would rec,uire access to the original vtdeo, „o„.t thts protected verston of it 
In a second example, an author wants to have a presentauon w„h a free anmct 

presentatton, which is organized as a set of "acts" 

SD graph update streams may be open in parallel. The time stamps on the ALUs in the 
streams are used to synchronize and coordinate. 

separate CMPO. There ,s hkely an addittonal SD-graph update stream that creates a simple 
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roo. nod. .ha. is ,„v,sib,e ™d s,,e„,. TOs node brings ,n U« o,her _„,s of ,he 

presentation as needed. 

Th. foregoing descnpnon of .mplemeMations of the ,„ven,i„n has been presented 
.or purposes of i„us.a.io„ and descrrption. ,s no, e.hans,ive and does no. i J„ Z 
_ ,0 ,h= preetse for. d.sCosed. Mod,.ea,ions and variations are posstbie ,n „gM 
of . e above teaehtngs or .a. be ac,.red practictng of.be invention^ For example 
d>e escrtbed ,n,p,e„e„tation includes soWe b„. .be present invention tnay be 
.ntplemented as a con,bi™„io„ of hardware and software or ,„ hardware alone The 
.aventton may be tmplentented wtth both objeet-oriented and n„„-„bjectH,r,e„ted 
progranrmtng systems. The scope of the i„ve„«„n ,s denned by the elatms and thetr 
equivalents. 
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We claim: 

a control arrangement including: 

, ■"=»^ft>rdec,yp„„gU,ee„CT,p,edportio„„f,hecoMeM 

compressed information, and ''"^^ """^^^^^ '-'"'^'"g 

wherem the player further includes: 
a demux designed to separate and route the sub-streams- 

mformation; and patnway for the transmission of 

3 The player of Claim 2, further ineluding- 

-.^^-P..hedee.p,ed.h.3,re:.lr.::~"^^^^^^ 
The player of Claim 3, further including- 

con.r»,ler. dec,yp,i„„ ^^^^J^^' "..^ s,re™ 

J ■The player ofClaim 4, further including- 

«;.c::nr.:rr,::r:3^^^^^^^^^ 

*n,.neat,„n of „,ee. .Meh are to he rendered or have J rendereT 

format. "'''^ '' ^^««"' - -oded in MPEG-4 
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fo^aJ' encode. i„ MP3 

12. The p,ay„ „f Cai. 8. wh..i„ „e rule „. ™,e ... 



aspec, of access ,o or use of the gove^ed sub-slream or objec. 

o..e.:Lr::ra::r^z''*"""^°^^^^^ 
.e.oj:e.:::r:™^^^^^^ 

govemcLh "'^'^"^ ^'e se, specifles ,ha, „e 

~::ro:::;:;::— 

=ap->.:::.»c::::rc:r::i::::r°""--^ 
~„o„po„a.e.a„.es,r::::::r::r '""-^"^"'''"^ 

.woJs;™tt™:eo^^ 

g s to or use of the same governed sub-stream or object 
The player of Claim 1, wherein ,he control arrangemen, ,„c,„des .anrper 



22. 

resistance. 
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23. A digital bit stream including- 
co„,e„, ,„f„™,„„ ,H„ „ 

a secure comainer including 

governance ,„foma„on for ,he goven,a„ce of a, te, one aspec, of 
-cess ,„ or use of a, leas, a pon,o„ of ,he conlen, ,„fo™.Uon. and 

■nfor.,:,!: ' ™ 

encode "n MPSO-rCla:" '"^ *™ — 

26. A niethodofrendering a protected d,giBlb„ stream ,ncl„d,ng 
receiving the protected digital bit stream. 

passing the protected digital bit stream to a media player 

.he media player reading first header i„fon.atio„'ide„ti^i„g a plu.m used 
-o process the protected digital bit stream, the firs, header mfolt;„ 
indicating that a first plugin is required; 

the media player calling the first plugin; 

■he media player passing the protected digital bit stream to the firs, plugin- 

necessary ,„ order to render the decrypted digital bit stream 
ihe first plugin calling the second plugin- 

2= plugin passing the decupled digital bit stream to the second plugm 
second plugtn processing ,he deeryp.ed digi.l bit stream, Ote processing 
ludi^deeompressing a. leas, a portion of .he deeryp.ed dig,., bit streal 

iedi:::::rd^^^^'"""^"'^^" 

-he med,a playe, enabling rendermg of ,He dec^pted and processed drgiial bi, 

whereby the first plugin may be used ,„ an archnecire no, designed for 
muliiple stages of plugin processing. 
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